 Let us prove two more languages to be regular, just to get our ideas about DFA and their capabilities fixed in our mind. These two languages are like this. The first language is gain strings of binary over the binary alphabet. Here, we want that the third bit from the left end is 1 and this other language similar looking, but you will see it is kind of fairly different in the implementation. Third bit of x from its right end is 1, so that two things are similar except here, we want some condition from the left end in the first case l 1 and here from the right end. So, let us say I have this string 0 0 0 1 0 0. Is this in l 1? It is not because the third bit from the left end. So, for l 1 you are interested in the third bit from the left end. This bit is 0, but it is not 1. So, therefore, this is not in l 1. On the other hand, the same string 1 0 0 0 1 0 0. This if you look at from the right end, the third bit is 1. So, string satisfies the bracket here. Therefore, this is actually in l 2. How do we design DFA's for these two languages l 1 and l 2? Well, l 1 should be fairly simple. So, what is the kind of information the finite state head should keep in its mind? Imagine you are looking at binary string starting from the left end. So, the first bit is not of interest to you, but you should remember that I have seen the first bit. So, when you see the next symbol, you know it is the second bit. Again that is not of interest except to the point, except to the extent that the next bit is going to be of interest to you. So, you will agree with me that consider this machine m 1 on either 0 or 1. It just goes to this state. So, basically this state means I have seen no bits of the string. When you come to this state, when the DFA comes to this state it remembers that I have seen the first bit from the left. Then when the DFA comes to this state that means it has seen exactly two bits from the left end and now the bit that comes that would determine whether the input is in the language or not. Now, if this bit is 1, the third bit 1, 2, 3. So, now you should accept the string and what more if that third bit is 1, then it does not matter what bits comes subsequently. It should remain in this state which is the accepting state, the final state. On the other hand, if this bit was 0, then it goes to a state. Now, any further augmentation on the bit string cannot make the string in the language l 1 because the third bit is 0. So, it gets into a trap state and both for 0 and 1 it remains here and that is it. So, I claim if this machine is m 1, I claim that l of m 1 is l 1 and you will see that you will agree with me. This is fairly simple to see. We are just this DFA is looking at the third bit from the left and deciding whether to accept the string or not depending on the bit is 1 or 0. Now, let us consider the language l 2. So, you see the one problem immediately we face that we cannot go to the end of the string and then start back to check what the third bit from the right end was. So, imagine the string. So, you cannot, the DFA cannot do this that it cannot go all the way to the end and then say the input has ended. So, let me trace back and check what the third bit from the right end was. So, what it should try to keep in its head, what the of the current string, let us see the DFA has seen. Let us see the DFA has seen an input string up to this point. This is the last bit it has seen. It is the last bit it has seen. This is the last bit it has seen. What is of importance to the DFA is whether or not that this bit is it 0 or 1. If this bit is 1 then if the string ends here then that string should be accepted by the DFA that we are trying to design. If that bit is 0 then that string is not going to be accepted. Problem is it is not just the third bit that you need to remember because you see let us see after this this is the part that you have seen of the input and this is not the end of the input. So, another bit comes then which is the bit you should remember or which is of importance to you is now not this bit because now because of the addition of 1 more bit in the input string this string that I have seen so far. This is the bit which has become of importance and then if you see another bit this is the bit which has become important. So, one can see it is not difficult to see if you pause a moment that if I keep track of all the 3 bits that I have seen at the end of the string that I am currently looking at. In other words supposing I have seen a string up to this point and what I should remember is the these 3 bits. Now to remember 3 bits how many different combinations of these bits can be there that you know there are 8 possible combinations because this can be 0 or 1 this can be 0 or 1. All these 8 possibilities must be keep track of separately because for example, this was 1 0 1 and this bit was different let us say 1 1 1 when the next bit came let us say 0 came. Then if this was the entire string then and this the string ends here then this string should be accepted the 1 below the 1 above should not be accepted because the third bit from the right is 0 here. So, let us see we keep track of our DFA will remember what are the last 3 bits it has seen and in the sequence it has seen right. So, in this case you can see it should must have 8 different states at least. So, let me and let me right here let us say this state means the DFA my what I am trying to ensure the DFA should be in this state if the last 3 bits it has seen is 0 0 0 last 3 bits 0 0 0 here it is let us say 0 0 1 0 1 0 0 1 1 here it is 1 let me put it this way 1 0 0 1 0 1 1 1 1. So, 1 0 0 1 1 1 1. What is the goal what is our goal for the DFA that we are trying to design that this DFA will remember the last 3 bits it has seen the sequence of last 3 bits it has seen if it has seen the last 3 bits that it has seen if it was 0 1 0 it should be in this state and so on. So, now let us think of the transitions right. So, now here a 0 comes. So, last 3 bits it is in this state if the last 3 bits were 0 0 0 and now a 0 came. So, which is the state it should go to on 0 clearly even now the last 3 bits that sequence is 0 0 0. So, it should remain here on the other hand if a 1 came last 3 bits it has seen will now be 0 0 1. So, this comes to this state now we have taken care of this state and so from this state what are the. So, 0 0 1 and now a 0 came. So, the last 3 bit we are talking of this particular state if the machine was here at some point then the last 3 bits it has seen the sequence is 0 0 1 now a 0 came. So, it is the sequence now it has seen 0 1 0. So, it should come here on 0 and on 1 it should go to 0 1 1 we have completely taken care of this state and now let us just consider one more state. So, if this is the state the machine was in that means the last 3 bits it has seen are 0 1 0. So, now a 0 came. So, the last 3 bits became 1 0 0. So, on 0 it should go to this state and if the last bit is next bit is 1 it should go to the state 1 0 1 that remembers 1 0 1. So, this is 1 to be the last 3 bit sequence and this way we can complete the diagram. So, this way when we complete the same things for all the states our transitions from the states will look like this looks somewhat club somewhat untidy, but conceptually it is fairly simple that we have realized. Now, but the DFA is not complete I need to say what are the final states that is easy a final state should be that state in which the third bit from the right end that it has seen so far is 1. So, for example, if the machine was in this state then clearly the last 3 bits and the string ends there and string ends there and machine finally finds itself in this state then that string should be accepted because the third bit from the right end then would be 1. So, you can see all those states in which the first of the sequence that we are remembering which is the third bit from the right end of the string that I have seen so far if that is 1 it should be an accepting state. So, clearly then this should be accepting state and so should be this and this and this on the other hand these 4 states are should not be accepting states all right do we need more states than this you might wonder that the meaning of the these states that we have put here is this that the last 3 bits that I have seen, but when I start right supposing when the DFA starts it has not seen anything. So, there is nothing like last 3 bits similarly when it has seen just the first bit it has seen. So, when it is here at this point having seen the first bit it has seen only 1 bit there are no 3 bits that it has seen. So, what we do for this you could have states for the initial part, but as it happens let me just say this that if you start here the thing is all right because although you have not. So, you have not seen the last 3 obviously when you are starting what you have seen is empty language empty string, but it is all right to think as if you have seen 0 0 0 because why because whatever happens the next 2 bits whatever be the next 2 bits are right next 2 bits will make next bit will make this bits to be the last bit another symbol comes in this will be the last bit these on no 2 bit strings the machine should accept because the last the third bit from the right is not 1. So, you know you can convince yourself this is all right to keep this as the initial state if I wanted. So, that that gives a that completes the description for a DFA for L 2. So, if I say this machine is M 2 then claim is language accepted by M 2 is L 2. Now, I can there is nothing very sacrosanct about the third bit supposing I had said the 10th bit from the right end this similar machine I could have designed, but if I use the same idea as was used here then how many states that machine will have which tries to accept all string whose 10th bit from the right end is 1 it will have 2 to the power 10 states 1024 rather too many. Now, you may wonder is it really necessary can we do for example, could we have taken care of could we have accepted this language L 2 using a DFA with less than 8 states. The DFA that we designed had 8 states 2 to the power 3 8 states because we are remembering all sequences of all 3 bit sequences which came at the end that is the end. So, far encountered and therefore, I have 8 states can we have done with less number of states and it is not too difficult to see and that for accepting L 2 any DFA needs at least 8 states. And in fact, if my language was instead of trying to check whether the third bit from the right end is 1 if it was 10th bit from the right end is 1 then I would have claimed that for accepting L 2 any DFA would have needed 2 to the power 10 which is 1024. Let me just prove this statement for L 2 that we need at least 8 states in this manner just consider these 8 strings 0 0 0 0 0 1 1 1 1 0 1 1 1 these are 8 strings the 8 strings of length 3 binary strings. And I am trying to prove this statement and I am trying to prove this statement that any DFA that accepts L 2 must have at least 8 states. And we will prove it by contradiction suppose m accepts L 2 and has less than 8 states. Now, remember the machine has this machine m has strictly less than 8 states. So, it has 7 or less states and here I have a list of 8 strings. So, what I do is this machine let us say m consider running m with these 8 strings. So, of course, it will start with its initial state m's initial state and let us say at the end it goes to some state q 1 here it is q 2 and so on. So, there are total number of states is 7 or less number of strings we have 8. So, now you remember from pigeonhole principle it means that there will exist 2 strings here which will take the machine from the initial state to the same state. So, can these 2 strings be like that that this string 1 1 0 and 0 1 0 took m to the same state. But that is not possible because this string should be accepted whereas, this string should not be and that state is either an accepting state a final state or an auto not a final state. So, it cannot behave differently that it should it will accept this and not accept this because the state is either final state or not a final state. So, clearly these kinds of things are not possible, but may be you may say that let us say this string and this string took the machine to the same state. So, let us say that is ok because both are final both are strings with the third bit is from the right end is 1. So, both should be accepted. So, that is fine. So, suppose it took this took this string took the machine to q and this string also took the machine q. One thing you can assert that q should be a final state there is no contradiction. So, far, but imagine now another state is bit comes. So, let us say another 0 comes now what is going to happen because here in both these cases the machine m went to the same state q and now a 0 came. So, again it will go to the identical state let us say q dash. So, q on 0 goes to q dash some q dash and now we have a problem because now what should q dash be is it a accepting state or a non-accepting state. Because what is the problem this string should not be accepted on the other hand this string should be accepted because this is this we have a 1 from the which is the third bit from the right end. And so we have a problem we have a contradiction for this I need q dash to be non-accepting non-final here I need q to be a final state and that is a contradiction. So, we got into the contradiction because we assumed that this machine m which accepts L 2 has less than 8 states. So, therefore, my conclusion is any D F A which accepts L 2 must have at least 8 states. And therefore, we can by the way you can this argument that I have given you can check that that will be the case whatever be the 2 strings which take the machine to the 2 3 bit strings that take the machine to the same state. If the machine had less than 8 states it has to happen the 2 of these must take the machine to the same state and you can derive the contradiction. So, if we had the language another language where we are trying to see whether the 10th bit from the right end is 1 or not. Then by the same argument if you are doing it by D F A that D F A necessarily will have 1024 states at least. Now, we have seen many examples of languages accepted by D F A. We know that every finite language can be accepted by D F A that means every finite language is regular. Then we have seen several examples of infinite languages which are also regular. We have also seen that if a language is regular then its complement is also regular. If 2 languages are regular then their union as well as their intersection both are regular. So, we have a fairly large class of languages we feel intuitively which is that class of regular languages is fairly large. At this point of course, you will wonder can all languages for which I can decide the membership problem by means of a computer program is it possible that all such languages are regular. Now, intuitively the answer is should be no why because you see suppose you see L is regular. In that case necessarily there is a D F A M to accept it. Now, suppose I am interested in solving the membership problem of this language which is a set of course. That means, somebody will give me an input x which is a string and I am both side if x is in L this language. So, this is set membership problem. Now, since L is regular there is a D F A to accept L and given x I can just run the D F A on x. Of course, you can vary the number of languages easily. I am not going into those details, but you can see you can easily write a computer program to simulate a D F A. That computer program will take symbol of the symbol of the string from the left end and implement the D F A to check which is the state the machine would be having scanned the some part of the string. The initial each for each initial prefix for each prefix it would the your computer program will figure out where the D F A will be. It is fairly easy to write that program and what you will be the time complexity and that program of course, will solve this problem because when the program at the end of the input string reaches a state of the D F A which is a final state then it would know the string x is in the language otherwise the string x is not in the language. So, this program and since you have studied some simple I mean some amount of algorithms and you know about time complexity. When you implement the D F A to solve this problem that algorithm time complexity is going to be order n where n is the length of the input here the input is a string. So, that means now if you thought if somebody tells you that look I think that all membership problems can be solved by D F A's then you know you will realize that there is something wrong because all membership problems if they are going to take order n time which which is too much to expect you know you expect that some membership problems will be harder even if these membership problems are computable maybe it will take n log n time n square time n cube time and so on. So, intuitively we feel therefore that there will be many languages which are not regular right even if we can solve the membership problem of such problems by computer programs and now we will argue we will give you a methodology for proving a concrete languages to be not regular. So, so for example we will prove we will show that this language L which is 0 n 1 n. So, basically these are strings where you have a number of zeros followed by number of ones only constraint is that that the number of zeros is equal to the number of ones. So, 0 0 0 1 1 1 that should be in the in this language L on the other hand 0 0 0 1 1 that this should not be in the language L this should not be in the language L because the number of zeros here is 4 here it is 2 how do we show in fact how do we show that a language to be not regular. That means I should be able to prove that there is no DFA to accept let us say this language and one standard method you can imagine will be that I will prove this statement. So, we will show I did not complete the sentence we will show that this L is not regular. So, proof method could be likely to be that assume L is accepted by a DFA because if it is regular then. So, we are we are we are giving a proof by contradiction and there will assume that L is accepted by a DFA we assume that L is regular. So, therefore, L is accepted by a DFA and then we should try to reach a contradiction well it is we we we we we we can intuitively see may be this thing will work and indeed it works this this proof methodology will work out and let me give you the intuition behind such proofs that see there is one severe limitation once you say L is accepted by a DFA comes about in this manner suppose L is accepted by a DFA let that DFA some case states. Remember that DFA can have any number of states, but once you say that this DFA accepts this language. So, you have fixed a particular DFA and that DFA has you know number of states which is a constant. Now, the DFA which accept some language L it has some case states and imagine there is a very long string which is in the language in L and consider the sequence of states the machine goes through. So, remember that if you had supposing these are the symbols of the string you will start with q 0 and then let us say q i 1 q i 2. So, seeing the first two symbols it is in the state and so on and here is another state and all that. So, we talked about it now if that string had let us say 2 the length of the string is 2 k and so this state sequence is going to be therefore length of that state sequence is going to be 2 k plus 1. So, then what that means again by pigeonhole principle that here is the sequence which is much which is the length of that sequence is larger than the number of states and each element of that sequence of states is of course, a state what it means therefore that some two states must be identical because we just have do not have so many different states to put states differently when the string is large. So, imagine a situation like this that I have a string and first part of the string. So, this is the string I am thinking in three parts that this part took the machine from q 0 to some q and since the string is long some states are going to repeat. So, let us say this state is repeating that is first part of the string let me call it u took the machine from q 0 to q second pair part of the string v let me call it v took the machine to the same state q because the state is repeating and then from here it go went to a final state. So, and let me call this w right. So, the way I have drawn this picture is it clear. So, in fact let me use this that delta hat of my machine m is let us say q sigma delta q 0 f. So, what we are seeing through that picture is that delta hat of q 0 u is q delta hat of q v is the same state q and this is the state q. So, this and delta hat of q w is q f some q f which is in the set of final states. Now, clearly the string u v w takes the machine from q 0 to one of the final states. So, u v w is in the language f what about u v v w look at these three things u takes the state of the machine from q 0 to q v takes the machine from q to q the first copy of v second copy of v again it is a deterministic machine it cannot remember that whether it is the first copy or second copy. So, it has to behave identically on both the copies. So, on this copy also takes the machine from q to q and now w comes. So, this machine also will take the machine take the this string will also take the machine from q 0 to q f. So, therefore, this string u v v w also will be in l and so on u v v v w also will be in l and so on. So, we can pump the string in which state is repeating on which a state repeats in such strings can be pumped to get larger and larger and larger strings all of which will be in the language if this string is in the language. Now, that is actually a severe limitation and we will see it how that is used see this intuition that I just told you is captured by a lemma what is called pumping lemma which is invariably useful for proving languages to be some non regular. So, let me state the lemma first and then before proving with the proof ideas like this let me first I will give you an example usually we use a particular result known as pumping lemma to prove some language to be non regular. So, let me state the lemma pumping lemma for regular languages the statement is something like this suppose l is regular then there exists a constant k that depends only on the language l on the language l such that for all x x is of course a string for all x in the language length of x being greater than equal to k greater than equal to k we have there exists or let me let me state it like this for all x in l the length of x greater than equal to k there exists u v w which are again three strings satisfying condition one is that u v w that is the concatenation of the three strings u v w is actually the string x second that length of u v is the length of the string is less than equal to k and the length of v is strictly greater than 0 that means v is a non empty string third this is where the name lem pumping lemma is coming from for all i i greater than equal to 0 u the string u v i w is also in the string u v i w is also in the language a couple of things when I say that this constant k that depends only on the language all we mean is that from language to language of course this constant may be different however once you fix a language this is constant that means it does not matter about the string that you are talking of and notice that remember our what we said right in the beginning that v to the power 0 also makes sense because i is greater than equal to 0 v to the power 0 is empty string that would mean basically we are saying that the string u w is in the language as well as u v w is in the language which of course begin with it was but now with i equal to 2 u v v w is in the language u v v v w is in the language. So, another way of stating this is that you can pump this v any number of times 0 or more times and yet the string that will get is going to be is guaranteed to be in the language l right before we prove this lemma let me show an example of its use l language we talked of little earlier set of all binary strings such that x is of the oh just say this language let me prove this using the lemma pumping lemma the way these proofs will go like this let l be regular we are I will prove this essentially by contradiction assuming l is regular then I will invoke this pumping lemma to get a contradiction let he be the pumping lemma constant consider the string x which is k 0's and k 1's now you can invoke the pumping lemma for the string because length of x is greater than equal to k in fact the length of x here is 2 k. So, the pumping lemma says that there exist from pumping lemma from the lemma there exist u v w satisfying these three conditions the three conditions of the lemma. So, this is your string there are k 0's here and k 1's here and now the what this lemma says that this string I can look at as a concatenation of three strings u v and w now the entire u v part that is concatenation of u v is less than equal to k that means where see this string I am saying is same as u v w do you therefore see because of this condition v u v together must be somewhere within 0's only and 0's because you have k 0's and length of u v is less than equal to k. So, in particular condition 1's conditions 1 and 2 imply that v consists only of 0's because u v itself consists only of 0's and therefore v necessarily will have only 0's and what more v is a non-empty string v is a non-empty string now consider the string on pumping 0 times that means let us see we just remove v. So, I will get u and w according to the lemma this u and w is in the language what did I do there was some v which is non-empty which I removed from this string of 0's they were k 0's to begin with now on removal of v length of v is at least one. So, clearly in u w can I claim this is clear in u w the number of 0's is strictly less than the number of 1's right 1's did not change because all we had removed was the string v and v consisted only of 0's u also consisted only of 0's of course, but so once we did not touch all we did was to take out a few 0's at least one 0 from this k 0's and now I have a string u w which the lemma tells me that will also be in the language but now that string has number of 0's which is strictly less than the number of 1's this string u w therefore, cannot be by definition of l l is the set of all languages in which 0's and 1's have the same number by definition of l u w this string u w cannot be in the language u w is not in the language. Now, your lemma says u w is in the language u w is in the language and definition of the language says u w is not in the language therefore, you have a contradiction and the way to resolve the contradiction will be that to assert one of the assumptions that you have made in the proof that is false and only assumption you have made in all this is that which could be false is l is regular. So, therefore, this proves that l is not regular. So, this is the use of this lemma we have seen one example of how to make use of the lemma to prove some language to be non regular we will show some more examples of the use of the lemma and then we will also of course, need to prove that the lemma is with correct. So, we need to prove this lemma and there are some simple variations of the lemma we should also talk about that we will do that in the next lecture.