 By now, you must be convinced that the Turing machines are quite powerful as devices that can implement algorithms. We have seen not just the basic model of Turing machines. We have seen several extensions that is adding of some extra power or facilities to the basic machine. We have found that even with added power, the class of languages which would be accepted after, for example, an extra tape is added, something like that. That class of languages which can be accepted by Turing machines, that class remains invariant. Before we study the limitation of this class of languages which we had called recursively enumerable class of languages. I would like to emphasize again that Turing machines are very powerful devices although they look very simple. By now, if you think a little, you should be convinced that Turing machines can simulate computers as we know. Let me just spend a couple of minutes on this point that now, when we say Turing machines can simulate computers, what we would like to say is that suppose you are carrying out an algorithm by means of computer as you know, that particular algorithm can also be carried out by a Turing machine. So, to convince you of this statement, what we can do is to consider a very simple processor. Now, again I would like to emphasize that just because I am considering a simple processor, I am no way limiting the scope of this particular statement because even a simple processor can simulate that is known fairly or however, elaborate processor that you might think of. So, what is a processor? Simple processor that can be modeled this way that is simple processor of course, executes programs, it has memory and this memory contains both data and the program and a simple processor can have some registers and registers also can store some bit pattern. So, you just talking of simple processors. Now, from our knowledge of computer organization, we know how does a simple processor operate? Because what it does that at any given time, there is some contents of program counter which is a register. Now, program kind counter tells me tells us instruction is to be executed next. Of course, you know program is a sequence of instructions, it can be of course, the execution of a program, how it will execute in which sequence these instructions will be executed that can depend on data, etc. That all we know, but basically there is some place which has the address of the next instruction to be executed and then of course, there is memory and what is the essential part about the memory? That every memory cell has an address and in that address, we that cell at that address contains some bit patterns. So, a typical instruction could be like add m 1 and m 2, m 3 something like this. This instruction is telling that add the contents of memory 1, cell 1 to memory 2 and the result should be placed in memory address 3. So, one way of doing all this by kind of machine that we have by machine I mean a Turing machine that we have, we can imagine our Turing machine has several tapes and one tape contains or models the memory that models memory and this also we may have you know we have may have another tape to do work, etc. We can say work tape just consider how an instruction like this could be added. First of all, we need to say little more about what we mean by modeling memory. So, we may see that we have some special symbol followed by a string and then again this special symbol followed by some string and then again here let us see in between I have a cell which also contains another special symbol let us say to imagine this. Now, what I will interpret that this part is the address after all address is a string and this part between sent symbol and the next sharp symbol is the we are seeing what would be the contents of the memory cell whose address is this. Now, of course in normal day to day processors we have some bound which is the word length etc that kind of situation but here in this model we can have arbitrary sized contents. So, now as an instruction which is to be executed which is like this let us say which will that you know these parts of the instruction will tell me 3 addresses. So, imagine that here I had the contents of the addresses m 1 is an address m 2 is another address m 3 is another address. Now, let us say this is the address 1 which is here m 1 now if you see here 3 addresses and here there is the code for or you know op code basically that is why we that is what we say op code for let us say addition which is a particular bit pattern. Now, what we the or Turing machine can do that in another tape it can fetch the contents of this address 1 how will it do so because you see this is a bit pattern and therefore, this of course goes to infinity may be on both directions. So, it looks for this pattern in the places in those places where addresses could be stored see that addresses are stored flanked by a sharp symbol and a sand symbol. So, it will look for it will look for such a such strings in this tape and compare with this bit or you know symbol by symbol if they match then it knows that it has reached the corrects address and then it can copy the contents here in another and similarly it can so basically what I am seeing is that it is not difficult to see that fetching the contents of some specified addresses can be done fairly easily in this way and now what does op code addition involve you know executing that it would just involve that adding this and this if the contents of address 1 is here contents of address 2 is here. So, essentially we need to add and you can imagine that depending on whatever is the op code here the Turing machine which is of course is going to be a big thing it will go to a particular part of it. So, you know after all op code is again a string the machine can decide you know to go to a part of its machines program or you know after all a Turing machine can be seen as lots of states and in each state it is we are being told what is to be done. So, it goes to an appropriate part of itself where these operations will be carried out essentially the operation what for addition is add this to this and may be the result of the addition we may you know we can carry out that addition in another tape and that addition is placed here contents of the result of the addition and now the last part of this executing this simple instruction was to store the result in the address in the cell whose address is address 3 again what we do is will be scanning the memory tape the Turing machine will scan the memory tape it will find out where this match is occurring again remember that strings which are flanked on the left by sharp symbol and on the right by send symbol these are addresses and when it finds then after that send it should place this one. Now of course, it might mean you know creating space or squeezing some space, but all those are simple operations by now you should be convinced that they can be done. So, after that it should carry out the next instruction. So, what would the next instruction where will the next instruction be next instruction will be also there in the memory somewhere at a cell whose current address is the contents of the PC. So, basically if PC was you know PC you can think of is somewhere again on some another work tape or something with the program counter. So, now these are all some tapes, but this PC is program counter contents here. Now normally what we mean is unless the program counter value changes because of the operation of the previous instruction we will add that is what we say add 1 to the PC which really means go to the next instruction. So, you know again you look for this address between sharp and send wherever you find it skip the next contents part go to the next thing here. So, whichever is the way either you know we can say physically that the instructions will be one after another on the tape. So, if you know you just carried out instruction corresponding to this address or contents of this address for the next you need to the address will be even here. So, that you copy and that will have the correct PC value that is what you do after this and then from there from that address you fetch the next instruction in this tape and that then you will be updated the Turing machine will be updated to carry out the execution or the simulation of the execution of the next instruction. So, this is at a very gross level I have described that to you or I have tried to convince you that it is not difficult to see that a simple processor can be simulated by a Turing machine essentially simulating each instruction of the this simple processor program in an appropriate manner. Now, therefore, what we can see that Turing machines are of course, fairly powerful in the sense that they can do all that our ordinary or even extraordinary computers can do and when I say that it can do whatever computers can do if you want to you know if you want to work on this statement in the sense if you want to find out what are the limitations of this statement in the sense that can Turing machines do everything that you might think of doable or it has its limitations there we need to be a little careful and therefore, we will come back to the context in which we operate and that is recognition of languages because that a language is some subset of finite strings over an alphabet and let us see the alphabet this is the alphabet of languages under consideration and now we may ask a very simple question that does some Turing machine exist a language let me put it slightly you know this is not very nicely put the way we would like to say is that you see there are a language over sigma is of course, a subset of sigma star right. So, the focused question is given any L which is a language over sigma you can ask this question does there exist a Turing machine to accept. So, in other words if you see what we are really saying is that we are asking the question are all languages over sigma recursively enumerable right that is the focused way of asking about the power of Turing machines. So far as language recognition is concerned over some fixed alphabet remember just once more the definition of recursive enumerability we say language is recursively enumerable if there is a Turing machine to accept that language and the question that we are asking is there are so many languages over sigma star do you have for each language L do you have a Turing machine M such that L is the language accepted by this Turing machine of course different Turing machines will accept different languages that is fine or the same language may be accepted by different several Turing machines. But given a Turing machine that accepts a specific language and we want to know that the class of all Turing machines when we consider the languages they accept the language languages the class of languages accepted by the class of Turing machines does that class exhaust all languages over sigma right. So, this is the question and answer to this question is no. So, there are in other words what we are saying that there exist languages over sigma which cannot be by any you can see that if we prove this statement then we have proved limitations of what Turing machines can do so far I have been trying to emphasize that Turing machines are very powerful objects in the or devices in the sense that they can carry out all kinds of algorithms. And as is the philosophy of this course we would like to understand the power limitation of the power of the of any device that we are considering. So, in particular we want to know that what is the power or where what is the limit up to which Turing machines can be used all right. So, this proof of the statement these two are I mean this statement that there exist languages over sigma which cannot be accepted by any Turing machine proof of this statement is by something called a kind of argument which is called diagonalization proof is by diagonalization argument this is a very powerful style of proof now you know some proof strategies like induction proof by contradiction etcetera this is kind of this is a special kind of proof by contradiction. And of you might have seen the proof that there cannot be a bijection between the set of natural numbers and the set of real numbers that proof typically that we know of is by a diagonalization argument I would like to explain the diagonalization argument first by means of a simple but a fairly powerful proof of now what I will prove is this statement that there cannot be a bijection between a set X and its power set which is normally denoted as script or P of X now and we read it as power set of X now what is the power what is the power set of X. So, P X is by definition set of all X 1 such that X 1 so power set of X consists of the set of or power set of X is a set itself as the name suggests and this set contains all subsets of X. So, in particular it contains the empty set because empty set is a subset of X as well as the X itself because X is again a subset of itself that this statement says that between X and P X no bijection is possible what we are trying to do let me just say it here example diagonalization our diagonalization argument will prove this statement which is of course same as that statement now as I said the diagonalization argument is also a kind of proof by contradiction and any proof by contradiction starts by assuming the negation of what you want to prove right here between X and P X no bijection is possible that is your statement which you would like to prove. So, negation of this statement is that there is a bijection between power set of X and X assume there is a bijection between P X let the bijection be B F and this F maps X to P X being a bijection of course F inverse also exists and F is 1 to 1 now what we will do is we define Y which will be also a subset of X and the definition of Y is as follows that Y consists of X in X such that X is not an element of this looks a very knock was statement but it is very interesting statement we will see how it manages to this definition itself how it manages to prove this very general result alright. So, essentially how we are defining Y this capital Y we are saying look an element of X will be in this set Y if that element is not an element of the set or subset of X to which the small X is mapped to this is what it is right now we can see clearly Y is a subset of X because Y by definition basically some elements of X. So, capital Y is a subset of X now since it is a subset of X mapped F will map it to of course some element X X for every X we have others but you see because it is a bijection F is a bijection the at this remember this F is from this set to the power set. So, since F is a bijection for every element of this set there would be some corresponding element of X which alright is the map of we are taking that element from here in other words for our so far as our purpose is concerned again Y is a subset of X since F is a bijection therefore it is onto map for its range and of course the range is the whole power set and in the range Y is there. So, there exists some let us say X prime such that F of X prime is this set I want to emphasize again that the existence of this X prime is because your F is an onto map on P of X and capital Y which you have defined in this manner is a subset of X and therefore it is a element of power set of X and since F is an onto map on this set there would be some element of the domain that domain of course is the set X itself capital X some element X prime of an element of X such that F X prime is Y alright. So, this is straight forward from the use of our definition of bijection and now consider the question is X prime and element of Y right X prime is a specific element of capital X and Y is a subset of capital X. So, it is a meaningful question that is this element an element of Y now suppose there are two possibilities either X prime is an element of Y or it is not an element of Y we are taking the first situation suppose X is an element of Y. So, now what does it is what is it saying that remember we are saying since X prime right we are asking this question is X prime an element of Y. So, X prime assume this so X prime an element of Y which of course Y by definition is F of X. So, I have X prime is an element of F of X prime, but if X prime is an element of F X prime it cannot be in the set Y because in set Y you are only choosing those X s which are not elements of the set that X is mapped to by the bijection. So, this really implies that X prime is not an element of Y. So, starting from the assumption X prime is an element of Y we are getting to the fact or the conclusion that X prime is not an element of Y. So, this is a contradiction. So, therefore this supposition is wrong the other what is the other supposition that X prime is indeed an element of is not an element of Y right here we assume the X prime is an element of Y and here we are saying X prime is not an element of Y. Capital Y is by definition F X right. So, therefore we have X prime is not an element of F of X prime. Now, if X prime is not an element of F X prime in that case you just look at this definition for Y this implies that X prime is an element of Y. Now, here again the situation is starting from this supposition we come to the conclusion which is just the negation of that supposition. So, there are only two possibilities either X prime is an element of Y or X prime is not an element of Y in both cases we are reaching contradiction which therefore means that our original assumption that there is a bijection F between the set X capital X and its power set that must be false. So, this whole thing implies F cannot exist and therefore we have proved that there is no bijection between X no bijection between X and its power set alright. If you see what is really happening in this proof the crux of the proof is in the definition that we made for Y and this definition made sure that no so basically the definition made sure that Y cannot be the map of any element X right. This is what we do did now this statement that Y is not or cannot be the map of any element X you may think of I am just trying to explain how did you get to this definition and because even what you are Turing machine context will use similar arguments we need to be clear about the way these proofs diagonalization proofs work. So, this is something you are trying to prove that construct some Y such that this Y cannot be the map what was the map we said F the bijection map of any element of X. So, basically we wanted to show that take any element X of and we wanted to ensure that F of X is different from Y right. Now, these are two sets right F of X is a set because F maps is so F of X is a set of elements from X and this is also a set of elements of the way we achieve for a particular. So, here again consider a particular X let us say X 1 and we want to ensure this condition that F X 1 is not same as Y. So, this statement is will be correct if for every element X 1 we manage to do this. Now, two sets are unequal can be two sets can be unequal because either this has an element which this does not have or this has an element which this does not have right that is how to we make we can say two sets are not same. So, the you can imagine the witness of this statement will be an element which is in the symmetric difference of these two sets right that is which one that element which is either here and not here or which is here and not here and interesting thing that we did that we made X 1 itself to be that element to be that witness. So, let us write this sentence fully witness of this particular statement will be an element which is in that means all the elements which are in F X but not in Y union. So, basically what is this say that this part consists of all elements which are in F X 1 but which are not in Y and this consists of all elements which are in Y but not in F of X 1. So, now this entire set if it is non-empty that would mean this non-empty would imply this particular assertion that these two sets are not same and the way we have defined this set Y that we make X 1 to be that element which is in the symmetric difference right how because you take your map right and see what X 1 gets map to this is a particular set. So, we will just call it Z right. So, F X 1 is of course a subset of capital X which we are calling Z right. Now, this element take this element X 1 we are asking this question is it in Z right if it is in Z will not put it in Y right. So, in that case if X 1 is in F of X 1 in that case X 1 will be here but not in here because by this definition since X 1 is an element of F X 1 will not put it in Y. So, this set will be non-empty because X 1 will be there or if X 1 is not an element of F X 1 then we will put it in Y. So, this part will be non-empty in either case X 1 is the witness of this. So, we are taking every element and making it a witness of the fact that our constructed set is different from that subset of X to which X 1 gets map to by our assumed bijection and thereby we are you know proving that this particular statement is true. So, again in summary what is the summary of what I am trying to say here if you see really a diagonalization proof you will see it is trying to prove non-equality maybe of two sets and this non-equality like here we said that you know we tried to prove whatever be the statement that assertion we broke we could break it down to maybe an infinitely many simple assertions like here that for any element take an element X 1 make sure that F X 1 is different from Y and each one of these you ensure in defining the diagonal set this Y will be called a diagonal set alright. So, we will see as a you know when you see for the first time such an argument it looks complex, but if you go back examine it once more in the light of what you wanted to prove and the basic strategy of the proof. So, you wanted to prove in this case that there is no bijection between a set and its part set and you your proof idea was assume there is a map and then you constructed a particular subset of X such that it was not in the range of F. So, by that what how did you do that that this Y is different from the map of every element. So, all these conditions one by one or you know conceptually you can see that for this condition X 1 is the witness for another let us say X 2 this X 2 itself will be the witness and so on. So, let me just complete that in this case that that element which is the witness. So, this is an example of a diagonalization proof this is fairly complex proof if you see if you are seeing it for the first time and it is a very powerful as at least this particular example or example proof of diagonalization makes a very powerful statement we did not make any assumption about the set X capital X whose part set we are considering this X could be finite and even in infinite we are not saying that it is denumerable or not. So, any set or any set the statement stands that there cannot be a bijection between that set and its part set. So, now what is the statement that we would like to prove in this course about Turing machines is by this method first statement that will prove is that there are this is what will prove that there are non R e again the proof will be prove by and let me give you the proof first of all from a very overall point structure first of all we will see that you know our case is simpler than the previous one in the sense that here everything is at least our Turing machines the class of Turing machines will be enumerable. So, what I mean by that is I can line up all possible Turing machines all possible Turing machines to accept languages over sigma. So, this all possible Turing machines to accept languages over sigma they can be kind of lined up that means what I mean is I can the word is enumerate which is a technical word, but think of this way you it is possible for you to conceive of all these Turing machines to be in some sequence m 1 m 2 m 3 m 4 m 5 and so on right and each one of this Turing machines accept some language. So, the language accepted by this is of course, l m 1 this is l m 2 this is l m 3 and so on and now you want to prove that there exist non R e languages. If a language is R e to accept which is which is this which is a language over sigma then this has to be one of these languages is it clear because these are the only Turing machines which will accept languages over sigma and these are the corresponding languages which are recognized by each of this machine. So, essentially these are all the R e languages which are over the input alphabet sigma. Now in the same style that the way we created here a y and made sure that that y is different in some way in our way particular way what we are going to do here is to define a language will call it l d such that l d is not l m i for any i. So, we will ensure define a language l d and ensure that l d is not same as for every i right once we manage to do that if I can manage to do that then that would immediately imply that l d is not why because had it been R e this would be one of these. So, in particular let us say had it been R e it may be recognized by some m star which is somewhere in this line up. So, let us say this machine m j suppose it if l d was R e then they would have been a machine in this enumeration of all Turing machines to accept languages over sigma let that be that machine be m j the corresponding language would be l m j, but we would have ensured since we are doing it for all i we would have ensured l d is different from l m j and thereby we prove again by contradiction that l d is not added. So, two things that I need to do which will do in the next hour first of all prove this that Turing machines which accept languages over sigma that class of Turing machines can be effectively enumerated which means I can line them up one after another and some ordering. So, I can talk of i th Turing machine i plus 1 th Turing machine n th Turing machine in that ordering and if at all there is a Turing machine to accept a language over sigma that will come somewhere here in this line up. And then I would define this language l d which by definition will make sure that l d is different from all these l m i and of course, our l d will the way will do it of course it will be a language over sigma and therefore, l d is not an r e language over sigma this is for a fixed alphabet sigma we will carry out all this discussion.