 Let us recall what we did last time very quickly. First of all, we define the notion of when two states of a machine M to be called distinguishable. Let us write that definition. Two states of a BFA, P and Q. These P and Q are the two states are called distinguishable. There is a string, string Z, let us say, such that of the two states, delta hat P, Z and delta hat Q Z. So, of these two states, exactly one is a final state. In other words, if Z takes the machine from state P to a final state, then the same Z must take the machine from Q to a non-final state. And if that happens for all Z, if there is one string Z which shows this kind of behavior with respect to P and Q, then we say the pair P, Q is distinguishable. So, that is one notion that we did. Then, this is of course the definition. The other thing we did was that we outlined an algorithm to find out all pairs of distinguishable states. And then, we proved that the algorithm is correct. Today, let us begin by considering the time complexity of the algorithm that we had outlined earlier. So, you recall the algorithm what it did. We are talking of time complexity. At first of all, there was an initialization phase in which any pair in which one state is a final state and the other is not a final state. Such pairs we made them to be, we called them to be distinguishable. And they are of course distinguishable because the Z there is just epsilon. So, that was the my initial set of states that we definitely know to be distinguishable. And then, there was an iteration process. And in the iteration process, in each iteration we considered every pair of states which is not yet found to be distinguishable. And then, we, so that first was initialization. And second was iteration. And in iteration, what we considered was considered each pair of states not found distinguishable, not found distinguishable. And depending on a condition, we decided to add or not to add the set of distinguishable pairs. Without going into the details, you can see this. The initial phase, what did we do? We took every pair in which there is one state is a final state, the other state is non-final state. And how many such pairs can be there? If there are total n states, total number of pairs with the unordered pairs that are possible of these n states is of course n choose 2 which is order n square. This is the total number of pairs. And as you can see in the initialization in the worst case, it will be about order n square. And each iteration, what we are doing? We are considering one already not found distinguishable state, every one of these states and then checking a condition. So, in general, they can be order n square states which are not found distinguishable in the beginning of an iteration. And for each pair, what we do is a constant amount of work. If you go to the details of what we do, the condition, you will easily see that for each pair of such states which are not yet found to be distinguishable, we do something which is a constant amount of work. So, and how many such iterations will be there? Clearly, total number of iterations is bounded by again order n square. Why? Because in each iteration, we definitely add one or more pairs to the set of distinguishable pairs set. So, therefore, since there are totally so many pairs to begin with and in each pair, we are adding at least one. So, at most there can be so many iterations. In each iteration, we are doing order n square work. So, therefore, total time complexity is order n 4. So, order n 4 is the time complexity of the algorithm that we have outlined. I should also mention that if you do a little bit more, then this algorithm can be improved to order n square. Let me just indicate what is that extra thing that you might like to do so that the time complexity improvement happens and that is this that suppose in an iteration, I take a pair which is of course, not found distinguishable till then. Remember, then what we do is say we have such a pair P Q and then we consider one symbol after another and say for example, on A, P goes to P 1, Q goes to Q 1, the machine goes to Q 1 from state Q on A. Now, if this is not already in the set, we just discard it, but you see suppose in some clever way, we can keep this information that if ever this pair P 1 Q 1 is found to be distinguishable, then P Q also should be marked as distinguishable, is it not? Because whatever string that distinguishes P 1 Q 1, if you add an A in front of that string that would distinguish P and Q. So, keeping this idea, one can design an algorithm which will improve the time complexity to order n square. Now, at the end of the algorithm, you of course, have discovered all pairs of states which are distinguishable. Consider a relation R on again set of states, let me call it R 1 and we see that two states P R 1 Q, this they are related by this relation if and only if the pair P Q is not distinguishable. The first point is that this relation R is an equivalence relation, the relation this relation R 1 is an equivalence relation and this is fairly easy to check. Reflexivity is of course, true that P and P that pair is of course, not distinguishable, they cannot be any string clearly which takes P, the first P to a final state and the next P to a non-final state, it is after all the same state both of both the states are the same state. So, clearly it is reflexive and symmetry is also easy to see on what about transitivity. So, let us say that P and Q are not distinguishable. So, they are related by this relation and Q R S to P is not distinguishable from Q, this pair P Q is not distinguishable and Q S is also not distinguishable. Can it happen that P and S, this pair P S is distinguishable, if so of course, in that case the relation R 1 is not transitive, but suppose P and S are distinguishable, then that means what let us say that there is some Z such that delta hat of P Z is a final state and delta hat of S Z is not a final state. This is one possibility P S is not distinguishable, is distinguishable means one such case will happen, but you can easily see that if P is a state which takes the machine or which from which the string Z takes the machine to a final state, then it must be the case that delta hat of Q Z will also take the machine to a final state in fact to the same final state or to at least this to one of the final states. Why because suppose this is not true that P Z takes the machine from P to a final state and that same Z takes Q takes the machine from the state Q to a non-final state, in that case the pair P Q would have been distinguishable. So therefore, I can say that delta hat Q Z must be in final state one of the final states and now use the fact that Q is not distinguishable from S. So, this actually also mean delta hat S the same Z because of the same argument takes the machine to a final state, but that contradicts this assumption. So, therefore, the relation R 1 also must be an equivalence I mean it is a transitive and therefore, the relation R 1 must be an equivalence relation. So, now what we do is given the set of states we use this relation R 1 to partition the set of states. Remember an equivalence relation will partition the set of states into a number of equivalence classes and what we will do is that for each equivalence class of this set of states will have one state in our machine the machine the best machine that we are trying to design and justification for this should be apparent from our discussion on the Mahill-Neroded theorem. So, essentially what we are doing using our algorithm first of all we found out all pairs which are distinguishable from that set we also came we know it is easy to see which all pairs of states are not distinguishable that when used as a relation is an equivalence relation that equivalence relation partitions the set of states in equivalence classes and for every equivalence class of such states we will have one state in the final machine that we are going to build with a with a proviso which I will come to later, but at this point it is good to take up an example and see what is happening for the entire process. Consider this machine this DFA which we call M and we would like to minimize the states of this machine M to come to another DFA with minimum number of states which should accept the same language. Now, this looks complicated, but we can argue about what is the language accepted by this first of all notice that the initial state is this A and then on one you go to B from A on another one you will go to C and D and back to A and on on 0 from any such states you will go to another state and then come back here like here you go from B you are going to F on 0 and on from F on 0 you will come back and so on. So, and you will accept if at the end of the string the machine is either in A or C. Now, what is this machine trying to remember? I can see that this machine is trying to remember what is the number of ones modulo 4. See for example, if the total number of ones the machine has seen in an input so far if it is in this state you should be able to see that that number total number of ones will be when you divide by 4 the remainder will be 3 and actually the same thing here also because you come to these kind of states these outer states only on 0s from one of the inner states. So, here the number of 0 ones seen is 0 modulo 4 here it is 1 modulo 4 2 modulo 4 3 modulo 4 and so on. And here the same thing the number of ones seen here again is 0 modulo 4 but what about 0? In 0s you are just trying to remember that what is the total number of 0s I have seen whether it is even or odd and it is if you if we consider it a little carefully it is quite clear because we are accepting here and here that means the language accepted by m will be the set of all strings 0 1 star such that number of 0s and number of ones both are even. But of course for this language we had given right in the beginning one of our first or second class we had given a very simple 4 state automata 4 state DFA. And we also proved that time by an ad hoc argument that there can be no DFA accepting this language which will have less than 4 states. So, in that automata that we had provided that automata already had minimum number of states. Now that we have outlined the Michael Nerode theorem etcetera can we use whatever we have learned to go from this DFA to that earlier DFA of 4 states that we had outlined long back provided long back. Now first of all remember our first step in doing this minimization is figuring out which all pairs of states in this are distinguishable for that one way of writing that will be you know suppose I have there are 8 states a b c d e f g h. So, let me make a table like this this is just a way of keeping track of all pairs without bothering about their order a b you see what I mean is table like this. What we will do is we will put a mark for example, if we find by our algorithm that the pair a g or g a equivalently that pair is distinguishable then we are going to put a cross let me just keep the stable. So, what is our first step our initialization for finding out the all pairs of distinguishable states it is to say any pair in which one state is a final state and the other is not any pair if one state is final state and the other is not then that pair will be right in the initialization phase will be considered distinguishable. So, which are the final states a and c. So, a and c. So, a therefore, is distinguishable from h from g from f from e from g and of course, from b and similarly b is distinguishable from c right because b is a non-final state and c is a non-final state anything else that we can do. So, for example, let me write it in here. So, c and this b and here c is distinguishable from. So, c is distinguishable from h c is distinguishable from g c is distinguishable from f c is distinguishable from e c is distinguishable from d right and anything else that we can put. So, the c column I have completed and a column I have completed is and in a c column here the c whatever the column here that I have completed. So, these are the only crosses that I can put to begin the initialization phase. Now, we will start the iterations this was the initialization phase and in the iteration. Now, take a pair like let us say g and f right in the initial phase the initialization phase initialization will not we did not find this to be distinguishable. So, that is why we did not put a mark, but consider what happens that when we provide a 0 to this. So, just consider what happens g on 0 goes to c. So, let me write that here g on 0 goes to c and f on 0 f on 0 comes to b. Now, remember that this pair c b was already found distinguishable right in the initialization phase why in fact that is what you know because c was a final state c was a final state and b was not. So, we already knew that there was a cross here c and b. So, from here when I considered this pair g f in the next iteration on 0 I found this is the pair that results and there is already a mark there indicating that this pair is distinguishable. Then I will make in that iteration g and f to be this pair to be distinguishable. So, now I will put another cross here right and this way we will keep on going for example, let us see in the next iteration even to be. Now, remember that this pair c b was already found distinguishable right in the initialization phase why in fact that is what you know because c was a final state c was a final state and b was not. So, we already knew that there was a cross here c b c and b. So, from here when I considered this pair g f in the next iteration on 0 I found this is the pair that results and there is already a mark there indicating that this pair is distinguishable. Then I will make in that iteration g and f to be this pair to be distinguishable. So, now I will put another cross here right and this way we will keep on going for example, let us see in the next iteration you will when you consider the pair f and e and then consider what happens on the input symbol 1 you see f from f on 1 you go to g. So, this is and from e on 1 e on 1 you go to f right. Now the point I am trying to make is that having discovered that g f is distinguishable we had put a mark in the next iteration when we considered f and e this pair f and e then on when I consider the symbol 1 I see this is the pair already there is a mark. So, f and e will get a cross here and so on I did not give all the crosses that would have come in the first iteration. Now, when we complete all the you know all the iterations and when we go find an iteration in which there we have not added any extra pair into our set of distinguishable pairs. That means in an iteration when I could not put one more cross then the algorithm stops at that time this table will look like. Now, if you look at this completed result of our algorithm then what do we find that a and there is no cross in this position. So, we know that a and c they are not distinguishable because had they been distinguishable our algorithm would have put a mark there. Similarly, I know d and b they are not distinguishable and here it shows e and g is not distinguishable and f and h is also not distinguishable. So, now if you go back to our relation r 1 what do we say that we take the set of states of this machine in this case this set of states will have a b c. So, let me just write this way a. So, what we found was that these two will be in the same equivalence class and in fact they will not be any other state in the same equivalence class. So, a and c d and b f and h and g and e these are the four equivalence classes they have partitioned my set of states. And now what we are going to do is to consider an automaton where that automaton will have one state for each of these equivalence classes. We are outlining the process of building the minimized automaton with an example from this automaton. Now what we did that we had one state for every equivalence class. So, a and c now what will be the initial state of this new automaton that I am building. Wherever whichever equivalence class in which the initial state of the original machine was there. So, a was the initial state in this original machine. So, therefore, this is the initial state of the machine that we are building and now we have to fill up the transitions. So, let us see from this state as what which is the state the machine would go to on a 0. What happens on a on 0 it goes to e. So, therefore, it is sufficient to just consider what happens on one of these states and so on a on 0 this machine went to e and we check the equivalence class or the state in which the state e was there equivalence class. So, this is the state which corresponds to e g e is here. So, 0 we are here and so similarly from a on 1 where do you go a on 1 you go to b. So, we will put this transition here and now if you see from e where would you from this state on 0 where would you go to. So, let us just see from e on 0 where do we go to we go to a a is here. So, in fact we will come here and e from this state where do we go to on on on symbol 1 well from e you go to symbol 1 you go to f you know it should be obvious that instead if I had taken g here I just looked at on e where does the machine go on 1 and we said e on 1 the machine goes to f and. So, therefore, this is the equivalence class in which f is there therefore, we put an arrow here the transition on 1, but what could have happened if we had taken g from g on 1 where do you go to g on 1 we go to h. So, g on 1 we would have gone to h and actually would have put the same transition and that is not an accident or something if you think about it it that is what should happen and similarly from. So, this we have expanded the state completely and now from this state on 0 where should we go to. So, f from f on 0 we in this machine we come to be. So, f on 0 we will come here in this machine and f on 1 you go to or h on 1 both will be same h on 1 right you go to e. So, of course, you will come here and you should be able to complete it it will look like this and which should be which are the states here which should be called final states wherever a and whichever states in here equivalence classes where either a or c would find themselves. So, but as it happens here both a and c they are in the same state here. So, this is the only final state that the machine has now do you see that this machine is of course, exactly the same as the machine that we had provided earlier for the same language. And there of course, we had proved by ad hoc means that this is the best machine that we can do for this language and the purpose of the example is that we carried out the algorithm and came to an automaton. In general what may happen is that after doing this process you might find some states in this if I may call this reduced machine in reduced machine which are of course, with the reduced machine will have a states the equivalence classes of like this of the set of states of the original machine. So, if there is any state such that that state is not reachable from the initial state then from the so called. So, this machine that we are building we remove all those states which are not reachable from the initial state and the resulting automaton that will have will be the machine which is the best machine that you can have for accepting the language that we had. So, to complete let me just summarize the algorithm for minimization first of all you start with the process of finding out all pairs of states in the given automaton which you would like to minimize whose states you would like to minimize. So, you use our algorithm to find out all pairs of states which are distinguishable. So, once you find that then you make an partition of the set of states of the original machine and that partition will each equivalence class in the partition will consist of all the states which are not distinguishable. Then you will have a machine in which every state of the machine corresponds to one of these equivalence classes as a c here corresponded to this equivalence class this state corresponded to this particular equivalence class and then provide the transitions in the manner that we describe. And then finally remove any state which is not reachable from the initial state identify of course you have identified already the initial state to do this. Remember initial state is that state the corresponding equivalence class contained the initial state of the original machine and you also found out what are the final states these are all those states in this machine which corresponded to equivalence classes which had members of the final states of the original machine. And having removed the inaccessible or not reachable states the resultant automaton will be the automaton with minimum number of states accepting the language that you that this machine the original machine accepted. And what you can prove which is not too difficult that for this machine M if I call this machine as M 1 the relation R M 1 remember that there is an equivalence relation one can define using on of course the set of strings of the language of the alphabet this R M 1 this is actually the same equivalence relation which is R L that is these two are the partition sigma star identically. And therefore this machine will be the machine which has minimum number of states which is something we prove from or Michael Nerode theorem when we prove that the number of partition number of equivalence classes of R L has to be less than or equal to the number of equivalence classes given by any any any machine accepting the same language. And that that that actually prove that that any machine whose R M 1 partition sigma star in the same manner as R L does that machine has to be the minimum state machine the unique minimum state machine for accepting the same language. We have completed all that we wanted to say about regular languages in finite automata. So, this is the time for a very quick recapitulation of what we did in all these many lectures first of all of course we define the notion of finite automata and our first introduction to automata was the deterministic version of finite automata which we call DFA. And then we considered some variance NFA non-deterministic version NFA with epsilon transition we just call it NFA plus epsilon. At the same time right in the beginning we had defined the regular the class of regular languages by definition our language is regular if there is a DFA to accept it then what we found was of course it was easy to see that NFA is a generalization of DFA and NFA with epsilon transitions is a generalization of NFA. What we found that all these machines they accept precisely the same class of languages. So, all these corresponded to the same class of languages. So, given a regular language either you can make a DFA or an NFA or an NFA with epsilon transition similarly given from here you can go back to a regular language. Then we also studied regular expressions which is another way of expressing or denoting a regular language. Every regular expression we showed denotes a regular language and we also showed that for every regular language there will be a regular expression. So, this is yet another way of capturing the same class of languages which are regular languages. After that we studied closure properties and we found for all kinds of operations like union, intersection, complementation and then we considered things like reversal. For all these many many of these operations the language class is closed that means if I take two languages apply that operation to get another language that new language also will belong to the same class. Then we had also looked at some algorithmic issues like what will be the algorithm for getting starting from a regular expression and going to a DFA for example, which accepts the language denoted by the regular expression and so on. Also certain decision problems we considered that whether or not you know a language provided by a DFA or NFA or one of these is actually the empty language. Now somewhere right in the beginning we did another very important thing which was we provided a way of proving a language to be not regular. So, you see the point is if I want to show some language to be regular it suffices for me to provide a regular expression to denote that language or provide one of these automata to accept that language. But how do I prove that something is not regular? We gave a method we used you know pumping lemma to show that some you know certain languages are not regular. So, that is one method of doing it. And then finally in the last few lectures we looked at Myhill Nerode theorem showed how this Myhill Nerode theorem gives us a way of minimizing states of DFA. I should mention that no such efficient algorithm is known to minimize the states of an NFA. Of course you might say why do not you go from NFA to a DFA, but then that process itself in general can be exponential type. So, we do not know of any efficient method of minimizing states of NFAs. In fact, that ties up with big issue in computer science theory we will not talk about it now. One final point I would like to say that this you know this whatever we have learnt finite automata regular languages these are extremely useful in all kinds of branches of computer science. So, the point is that this this automata electron you will see that either this or these kinds of things or their generalizations certain generalizations that one people do they are very useful in many many branches of computer science. In particular you will see the finite automata they are used for lexical analysis phase in compiler. And I should also mention that there are many other things which in which we capture the notion that some process is there and there are only finitely many states. For example, I am I am not going to details here Markov process this is this tries to capture probabilistic processes which have finite number of states and there is not much memory. You see the whole point of a machine like finite automata that we have seen that the machine is in one of these states and that is the only thing that it remembers about its past that I have reached this state or that state no more. So, a probabilistic process which is basically uses that idea is called Markov process and then it is you know there are variants or other problems associated with it and their solutions are very very useful. For example, there is you might have heard of this term hidden Markov model these are used in speech processing etcetera. The notion of finite automata of course, used very heavily in formal verification of program. And therefore, this is something which is in extremely useful topic and a nice thing about this finite automata that almost everything that we would like to do we can do all the decision algorithms that all the decision problems they have algorithms. And as we go to other classes we will see more and more problems decision problems concerning such automata will become will be undecidable. That means, we will not be able to write any program to do the carry out those decision problems, but such problems are not there in general for regular languages and that is again another reason why it is so useful.