 From this lecture onwards, we will spend some time on this topic, Turing machines. Turing machines, the concept was defined by a British logician, mathematician, computer scientist whose name is Alan Turing and the definition came in a very famous paper that Turing wrote in 1936. Before we get into the details of what Turing machines are, how to define things with or work with Turing machines. We should have some understanding, we should have some appreciation, why did Turing define in 1936 what we now call Turing machines. The real motivation is to capture the notion of computability. What do we mean by to capture the notion of computability? Today, of course, we are very familiar with computers, programming, but at the same time, sometimes we ask that can all computational problems be solved by our computers. May be something that I cannot do today, something if at all there is something which I let us say cannot do today, may be even future when we have some other kind of digital computers or some other kind of computers, not electronic computers as we know today, but may be quantum computers, may be DNA computers. Can we then do something, can we write certain algorithms which could not have been possible to write by using today's computers? So, that means what all things can be done by computers. So, when I say to capture the notion of computability, essentially what I mean is I would like to answer this questions, this particular question, what all computational problems can be solved? This is the question we are trying to even today, this is the question that we have and way back in 1936 before digital computers were even there, Alan Turing defined this Turing machines and that today we understand that it totally captures the notion of computability. Now, if we understand what cannot be done by Turing machines and what can be done by Turing machines and then if it is indeed true that what Turing machines can do is precisely what any computer ever can do, then just by studying Turing machines will know the power as well as limitations of computability. So, now we can see therefore, this notion is of course, very important, but before again we discuss the details of Turing machines, one question arises and how is it possible that we define something once for all. In fact, it was defined way back in 1936 and the claim that we are making is that whatever be computers in future or today, they cannot do something Turing machines cannot do and whatever they can do Turing machines can do. So, that means we are capturing once for all the notion of computability, how is it at all possible that today we define something and we are saying that look this definition stands forever. So, far as a concept like computability is concerned, all right. So, I am just trying to tell you how not just important, how even difficult such a notion is because once for all we are trying to capture a notion like computability whereas, we know that computers change every day and yet we are trying to say that look what Turing machines can or cannot do that precisely defines what computers can do, all right. Of course, we know that computational problems are solved by algorithms. So, another way saying that if at all there exist an algorithm to solve a problem, that algorithm can be implemented on Turing machines. Now, can one give proof of this statement, what is that statement? What all can be done by any algorithm can be done, can be done by Turing machines. Now, if you notice supposing you say that I want a proof of the statement, what does it mean? Then of course, one needs to know that since proof is something formal, then one must understand formally what an algorithm is, but algorithm as a notion is an intuitive notion. Why do I say so? Because much before anything like digital computers were there to implement algorithms, we of course, had algorithms, right. For example, in way back in 300 BC Euclid provided his famous Euclid's algorithm to compute GCD of two numbers. There is no question of any computers to implement that algorithm then, but mathematicians down the ages, they have intuitively understood the notion of algorithms. So, point I am making is that algorithm is an intuitive notion and what Turing machine does is to capture this intuitive notion formally. So, let me say this Turing machine captures formally intuitive. Now, really speaking this statement cannot be proved, because as we shall see Turing machine is of course, is a formal concept in the sense that you can define it with great clarity. In fact, we will be making use of Turing machines to define many algorithms, but on the other hand this intuitive notion of algorithms as we said it is an intuitive notion. So, this kind of statement is saying that this formal notion captures this intuitive notion, right. So, really speaking we cannot have a proof of this. What we have is a thesis. So, this thesis is called Church Turing Thesis. So, let me write it here. What is Church Turing Thesis? Can be done algorithmically, can be done by Turing machines and vice versa. That is we can do. Other way is that what cannot be done by Turing machines cannot be done by any algorithm whatsoever. So, this is the so called Church Turing Thesis and very briefly I will just tell you why people have faith in this thesis. It is because at around the same time at around 1936 though many other not many other let us say 5, 6 other people in different countries. For example, Turing himself was in UK church in USA post in USA post in USA post in USA Gordel in Germany and Markov in USSR. They all independently tried to define formally this notion of algorithm. That is they were trying to see give a formal definition of what can be performed computationally. And they all actually gave came up with different model. Turing came up with Turing machines. Church came up with lambda calculus. Post came up with a certain rewriting system. Gordel defined what is known as general recursive functions Markov define Markov processes. They all look very different. And yet when it you know when people tried to look at all these things simultaneously. They found that the class of problems which can be solved by Turing machines is precisely what can be done by lambda calculus. Precisely what can be done by post system or Gordel's general recursive functions or by Markov. So, that does you know some kind of faith that this must be correct. And since then of course they have been some attempts by mainly philosophers of science logicians to very, very much elaborately examine and very critically see that is it is this thesis correct in the sense that may be that algorithms are more than what you know there are algorithms which could not be performed by Turing machines or vice versa. But you know all these attempts have shown that now indeed whatever you can think of to be reasonable definition of an algorithm you see that it can be performed by Turing machines. So, with this little background I would like to like to go into the details of what Turing machines are how we make use of them. Turing machines let me let me draw a diagram trying to describe what a Turing machine is. First of all what we have is a tape and we are already familiar with the notion of a tape. For example, we have seen finite state machines the input would be on the tape as well as for push down automator the input would be on the tape. So, this is a tape that Turing machine uses. So, there is a difference between the tape finite state machines use and Turing machines use you see this jagged lines on two ends they indicate that you should imagine this tape to be continuing to infinity in both directions once more that this tape continue to infinity in this direction as well as in this direction. And remember as before a tape consist of cells this is a cell this is a cell this is a cell and a cell can take on a cell you can have a symbol. Let us say a on this cell you can have another symbol and so on. Now, this is you can you can say is the tape of a Turing machine and there is this looks very much like finite state machine diagram that you recall. This is the if you like to call the control component of the Turing machine and it has a num it can be in a number of states. So, let us see it at any particular time it is in state q and this is its head this is the head of the Turing machine which is at this time scanning this symbol A. Now, in case of Turing machine this head not only can read the symbol which it is scanning, but it can also change the symbol. So, therefore this head that we have we would like to call it a read write head recall that finite state machines only had a head for reading a symbol from the tape, but Turing machines head reads the symbol and it can change that means it can write another symbol may be the same symbol. Now, how does it how does the computation or how does the working of Turing machine to be described. So, here to begin with let us say the machine is in some state q and scanning the symbol A. So, we can say present state is q symbol scanned in this case is A and now there will be a next state symbol written and move. So, let me just write this first. So, suppose the present state is q and the symbol scanned is A, then suppose here we have P here we have B and here we have R. So, you see there are 5 things q A P B R. What we mean is that whenever this particular Turing machine is in state q scanning the symbol A its next state is going to be P and it will write the symbol B in place of A. So, the symbol A is over written by the symbol B and then this read write head will move write by exactly one cell. So, from here if this description is to be followed the picture would be see that we are following this because at this time it is in state q scanning symbol A and next state is indeed P. Now, it has recall it has what here the symbol was A and now this symbol is B, but the read write head has moved in the next instant one step to write. So, this is called the move. Let us now be clear about what is happening I will repeat that at any given time a Turing machine is in a certain state scanning a certain symbol on the tape and depending on the present state and the symbol that is being scanned at that time it moves to a next state in the next time instant having over written the symbol that was there by this and it will make a move only by one cell either to the left or to the right in this case if this is the one it is following then the move is by one step to the right. Now, now you can see therefore I can describe completely the dynamic behavior of a Turing machine by listing down these kinds of quintuplets what a let me say what is it why we call it a quintuplet because there are five things here present state symbols can next state symbol written and the direction of the move right. So, dynamic behavior of a you know I will keep writing this T m abbreviation for Turing machine dynamic behavior of a T m is specified by the quintuples the T m. So, remember different Turing machines can have the set of quintuples different, but one particular Turing machine we associate a set of quintuples with that Turing machine and that will tell me if I am running that Turing machine what how will the Turing machine behave on a tape when something is written. So, by the way this set of quintuples for a particular Turing machine it is easy to see that this set is finite why because as before we assume that the alphabet that we work with is a finite alphabet right sigma is the alphabet which is a finite set of course as before and the important point is the number of states any particular Turing machine will have is also finite. So, Q is the set of states which is also a finite set and now we are not talking of non-determinism if you think currently only in terms of deterministic Turing machines determinism means that there is a definite move at each given time. So, how many quintuples can be there if the particular Turing machine has the set of states Q and the set of Turing machine the alphabet it works on is sigma clearly Q cross sigma not what I mean is if there are 10 state 10 states the machine had and 5 symbols and the total number of quintuples would be what at most 50 why I say at most I will come to that because may be that on certain state and for a certain symbol no quintuple is defined. So, in that case what happen supposing we had a state P and a symbol B but those no quintuple with state present state P and symbols can be. So, let us write it suppose a Turing machine has state P and there is a symbol B such that there is no quintuple with present state P. Present symbol can be supposing this is the case that means what that the machine if it ever reaches the state P and scanning the symbol B there is nothing defined in that case of course we say the machine holds in that case we say T m holds let us once more recapitulate whatever we are saying very briefly a Turing machine will have a 2 a infinite tape the tape going to infinity in both directions the tape is divided in cells each cell can contain one symbol from a finite alphabet sigma the machine can be at any given time in one particular state of a finite set of states which are calling Q and what happens when the present state is Q and the symbol being scanned is A is that some next state will be defined what symbol to be over written in this place that will be also given and the direction of the move remember that Turing machine move is always by one square either to the left or to the right depending on present state depending on the symbol being scanned some next state is defined some symbol what is to be over written is defined and the direction of the move is defined whether left or right and such a thing we called a quintuple and clearly the set of quintuples which we associate with a Turing machine defines the dynamic behavior that means what happens from you know instant to instant given that the machine is at a certain instant you know given doing something basically it is in a state and scanning a symbol then how its behavior will evolve over time that we can see. We also mentioned something about halting and that is actually a very simple way of defining halting that when will this whole process stop if at all ever it will stop is we can say that it will halt the machines you know this scanning over writing going to a next state and moving this kinds of behavior will sees in case it is in a particular state scanning a particular symbol for which no corresponding quintuple is defined so that is what we said here about that if the Turing machine finds itself in a state scanning a particular symbol for which there is no quintuple is defined in that case of course you know there is no defined activity and therefore the machine we can say as halting because it cannot do anything further and that is it that is the definition of Turing machine well you know we have to be a little more careful because we have said what happens from one instant to another but how did this whole thing begin in other words initially what is the situation. If we have a notion of initial state which you have seen in case of finite state machines right initial we say the machine is initially in its initial state and also in case of finite state machines we said initially the machine is scanning the left most symbol of the input this is in case of finite state machine in case of PDA we had one more besides this of course these things hold but we also had a stack and we also had to say whatever the initial stack contents will be now all those things we need to specify to see what is the case with Turing machines initially. So initially we will assume that Turing machine is scanning the left most symbol of its input which is written on the state somewhere. So let us say the input is A B A initially the Turing machine is scanning the symbol initially the Turing machine is scanning the symbol in a state called which we we are denoting as Q 0 basically this is the initial state of a particular Turing machine. So initially at time 0 if you wish the machine is in its initial state. So initially Turing machine is in its initial state scanning left most symbol and remember that input has to be always some finite string. So in this trivial example the input is a string of three symbols it is scanning the left most thing but what about the rest of the cells what is there initially in all those cells which are not being used by the input. So what do we expect and that is precisely what happens in our definition also that these are all blank imagine it is like a copy book when you start working on a copy book in the beginning the copy book case may be some problem is given. So that you know that is your input rest of the copy book is empty and that is what we mean by saying all the cells initially which are not occupied by the input string they are blank now of course what is blank blank is a special symbol. So blank I denote like this the symbol for blank is this what is the convention blank is a special symbol in the alphabet sigma why is it special because this blank symbol is what most of the input will have right initially the sorry the blank symbol is what most of the tape will have actually all the time but to begin with at least initially we can see the input is some finite string which is there the machine is scanning the left most symbol of the input and rest of the cells contain the special symbol called blank. So now let us at least give one example to convince you that it can certainly do at least more than what PDA is push down automata could do. Remember ultimately we would like to claim and we would like to convince you that this simple device that we are describing can do all computations but to begin with let us have one example where we see the power of Turing machines over push down automata. So let us take this problem the problem that will tackle now for as an for a for a for a for a simple first example of the capability of Turing machines is that that we would like to recognize strings of this forms what does it mean that imagine on the input we have a number of A's followed by a number of B's and then followed by number of C's. We will assume our input is such that that it has an indeed has a number of A's followed by a number of B's followed by a number of C's because in general input could have been anything but let us at least assume that input is of the form A star B star C star that is it has 0 or more A's to begin with then 0 or more B's then it has 0 or more C's. So this is the tape of the Turing machine initially will look like this and we would our Turing machine would like to make sure that the block of A's number of A's in the block of A's that number is same as the number of B's and the number of C's this the input starts here. So initially the Turing machine is scanning the symbol A in its initial state q 0. So how would such a device the Turing machine that we have described do this task of verifying that the number of A's same as number of B's same as number of C's. One way of doing this the strategy could be that C. So let us just see it sees an A and then it will make sure that corresponding to this A there is a B and a C. So what you might do or the Turing machine might do is that it will instead of A it will write a cross and then kind of move all the way when it sees a B it will put a tick. So that means corresponding to this A which has been crossed we have found a B which is now ticked and again it will move to the right to find the corresponding C and let us see it will change it to dollar. So after some time the tape will look like this of course these here we have blanks here also. So what it has done we will see how the details are worked out by a Turing machine. But basically it has found an A and corresponding to this A it has found a B and a C. But of course now it has to do for all other A's. So again so therefore the machine will come back make this A as the cross of course this cross exists. This remember this B has been already checked off because we took this B to corresponding to this A and therefore will tick the next B. And now corresponding to the second A will make this dollar to come in place of this C. So for example if you can see what is happening this simple idea that machine sees an A it remembers that it has seen an A. So therefore it must find a B and a C to correspond to this A. Why does it put a cross because it needs to remember that yes this A I have seen and I am now going to check off one B and one C. So basically it puts a cross for the A that it wants to make sure that there will exist a B and a C. So corresponding to that A the B that it finds it will put a tick and the C that it finds it will put a dollar and it will go on like this. So let us at least before we get into more details let us see that this can be at least this part that I am describing can indeed be done by the machine kind of machine that we have described. So let us say as we said initially the machine is in state q 0 that it is initial state. It is scanning this let us recall this is present state and this is the symbol present symbol and it goes to next state and of course remember the quintuple the symbol overwritten and move direction of the move. So in the initial what we said was initially as it scans an A it will overwrite that symbol by a by that A by a cross. Now should it remain in the state q 0 not really because this it now needs to remember that I have seen an A corresponding to which I must find a B and a C. So it moves to a let us say state q 1 and which direction should it move as we had seen that it will move to the right good. Now the machine will be in state q 1 and this kind of case you can see it will see A's what should it do with A's it should not change A's and it will remain in the same state q 1 and remember that A need not be changed of course because what in state q 1 what you are looking for is a B to corresponding to the A which has been crossed. So of course it moves to the right maybe then what will it find it will find sooner or later you know these block of A's will be over in state q 1 you are going to see a B and we said that this B will be checked off by putting a cross putting a tick there and it should move to another state because now it should find a corresponding C. So let us say it moves to state q 2 and it will still move to right then in q 2 it will see maybe more B's it should not change a state by seeing a B because in state q 2 what you are looking for is a C to correspond to the A which you have crossed. So B remains and it still moves to the right to find the first C or the C corresponding to the A that you have crossed in q 2 whenever let us say now it sees a C and it should move to a state q 3 by writing what should we do what should we do we have said for the C it will print a dollar and now what it should do it should now start moving to the left because this job of checking that for a particular A there was a particular B and a particular C that job is done. So now it moves to the left in this state q 3 and in state q 3 in general what is it going to encounter it will encounter not all you know as it is scanning now coming back you can imagine that suppose it has done this it has put this dollar and then it may see some more dollars it may see some more B's it may see some more takes it may see some more A's of course but it should start by crossing that A which is just next to the previous cross. So it should come all the way till it sees a cross and then move to the right so basically in q 3 if you see a dollar you print a dollar remain in q 3 keep move still keep moving to the left and in q 3 if you see a B you remain in q 3 do not change that symbol B till move to the left if you see an A you remain in q 3 keep it A till move to the left till you find a cross put a comma everywhere to that or let us remove all the commas since in q 3 now when you see an X of course that means you have found the right most A the last A that you have crossed off of course that cross remains in the and now what we should do we should move one step to the right that is the Turing machine should move one square to the right and start this checking for us A again right so basically it moves to this state q 0 right now you know of course you have not yet fully described the thing but already you can see that this is becoming a little too messy you know this writing this quintuples each one separately can we do a little better just for describing simple Turing machines as this yes we can use some diagram transition diagram for you know for Turing machines and of course that diagram is something it will be something similar to the transition diagrams of finite state machines but it should of course contain extra information like the direction of the move and if whatever the symbol being scanned what is the symbol being written because after all a move should describe a move as given by a quintuple all right so now you see one interesting thing about this example that look at this q 1 state q 1 right whenever in you are entering this q 1 you are moving to the right similarly whenever you are entering the state q 2 you are moving to the right we have entered q 2 here moving to the by moving to the right we have entered q 3 here we move to the left we have entered q 3 here again move to the left we have entered q 3 here we move to the left we have entered q 3 here move to the left and q 0 is the this is the only place we have entered q 0, we have moved to the right. So, this observation makes the task of drawing a transition diagram for a Turing machine simple, let us see how. So, at least in so far we had defined four states, this is q 0, q 1. Now, look at the first quintuple, in q 0 if you see a symbol A, you put a cross and why would not and go to state q 1. So, it is like this and this will be our convention that A cross that means in q 0, if I see an A, then the symbol that I write is cross and then move to state q 1. However, where is the direction? Now, we are now in this transition we are entering q 1, but one thing we noticed about this state q 1 so far that wherever whenever you enter q 1, you move to the right. So, why not please the symbol here right that means now you see that all the five things of the quintuple they described, if I am in state q 0 and seeing the symbol A, then I write the symbol cross go to this state by moving to the right. So, you are associating a direction with the state itself and the idea is whenever an arrow ends in that state, then that transition must be following the direction which is written inside this circle which represents R fine. So, let us go ahead and do this in q 1, if you see an A, what did you do? Again we move to the right by writing an A. So, essentially in q 1, if I see an A, I will write A and I will remain in state q 1. In q 1, if I see a B, then I move to state q 2 by writing a tick in that place of B and then I move to the right. So, therefore, of course, you will see that this is a right moving state in q 1, if I see a B, I will put a tick, move to the right and now we are in q 2 and if in q 2 if I see a B, we will remain in the state, but we will make a right move in q 2. If I see a C, what do we do? We go to a state q 3 because this C will change to a dollar and in q 3 when we enter, we have entered q 3 here. In all these places, we have entered q 3 and we move to the left. So, we can write the direction here and in q 3 if you see, so let us go ahead. In q 3, if I see a dollar, I keep it as such. So, this is a self loop really, but by make a move to the left. In q 3, if I see a B, I still remain in the same state, but I move to the left. So, I will just write like this. In q 3, if I see a A, still I remain in the same state q 3 and keep the same symbols there. So, A is replaced or rewritten by A itself and there we move to the left and now in q 3, if I see an X, then what we do? We go to q 0. Of course, we do not change that symbol. The same symbol is written back, but I move to the right. So, you can see that q 0 is a… This diagram actually, at least the iteration part of the Turing machine very clearly explained. So, let us just write follow it once. So, the iteration is what? Iteration is that given an A, starting with an A, checking it off with a B and a C and then coming, that is then coming back for the next one. So, let us just see here. So, the iteration begins the state q 0. So, in general, maybe I had a number of crosses and then a block of A's, then a number of ticks, actually equal number of ticks as you have put crosses, then some B's, then equal number of dollars, then some C's and then of course, black. So, you can now see this Turing machine working in iteration and this iteration is that take an A, make a cross and corresponding to this A, find the left most B that you can find, make it. So, if this will become a cross, this will become another tick, this B will become another tick, corresponding to this C, you will make it a dollar. And then you should come back and position yourself at the end of the iteration to start the next iteration here. So, and at the end of the iteration, you should be here. So, let us just see that is happening. Remember, in the beginning of the iteration, we are scanning this A and you see how convenient this particular diagram will be to understand this iteration. So, remember that the machine is here in state q 0 and it sees an A. Therefore, it makes it an x, which it has and now where will the machine be? It will be after making it a cross, it will be here. Now, here it sees an A, what should it do? It writes the same A, it will see all these A's, it will keep on being in the same state q 1 till it sees a B, this particular B, which will change it into this tick and move to the right. So, the Turing machine head is now where having put the tick, it is here. And then you can see it will, it is if it sees a number of B's, it will skip, but it will remain in the same state till it will see a C. But now, we can see that in this state, it will also encounter some dollars, but those dollars also it will retain and till it finds a C, then that C, it will change it to a dollar. Now, it will move left. So, when it will start moving left, this C after changing it to a dollar, it is in this state. See, it has changed it to a dollar, it is in this state, which we had called q 3. And in q 3, it is basically it is a left moving state till you find a cross and then you move one step to the right. Is that happening? So, in q 3, you are here, you have moved to the left having changed the C to a dollar. This particular C, it was changed to a dollar and now you have moved to the left. So, the Turing machine is going to see a dollar. So, what does it do on a dollar? It writes the same dollar, comes back to the same state by moving to the left and so on. So, it will after some dollars, it will see some B's, it will still keep moving to the left. It will see some ticks, should it do anything? No, not really. So, tick also, it will retain and it will see some A's. Again, it remains in the same state, it keeps moving to the left, because it is in this state, C is an A, writes an A and comes back to this state by moving to the right. It sees an A, it will come here, it will come here. Sooner or later, it will hit this A, left of which there is a cross. So, it will come back. So, having seen this A, it will come back to this state and now the state that the symbol that it will see is a cross. So, this cross is written back as cross and it moves to the right, canning A. So, this completes one iteration and therefore, you see what is happening in that iteration? What is happening is for each A, it finds a corresponding B and a corresponding C. Now, this iteration, if this successfully happens for all A's, that is for each A, it can find a particular B and a particular C, corresponding B and a corresponding C and at the end, no more B's or no more C's are left. In that case of course, number of A's is same as number of B's and same as number of C's. So, the Turing machine will be able to decide that indeed the block of A's and block of B's and block of C's that we had to begin with, they have equal number of symbols. Before we end, we will end this class here, but before we end, I would like to make a few points. Some of you already might be wondering that, is not this a kind of restriction that we are placing that whenever we are entering in a state, we always move in the same direction in this kind of diagram that is what is happening, but in general of course, that this one particular state, you can enter by either moving to the left or by or moving to the right. In that case, how can I take care of such a state in this kind of diagram? Indeed, you cannot. However, the point is that if you had a Turing machine, where a particular state is entered both by moving to the left as well as by moving to the right, then you can have an equivalent Turing machine, equivalent in the sense it will do the same job really, where each state when you enter will be like this, that is every state you enter on only one direction. So, we lose no generality by considering these kinds of diagrams as the diagrams for our Turing machines and this is a simple exercise to show a little later yourself, I am sure we will be able to solve that exercise. Given a Turing machine, which has a state or a number of states such that you know this state you move into, sometimes by moving to the left, sometimes by moving to the right, from such a Turing machine, you should be able to obtain another Turing machine, which will do essentially the same thing by having states with this restriction. So, that is the point number one. So, our point number one is our diagram for Turing machines is general enough. Second thing before we have not fully completed this example, but I think you are already convinced that indeed we should be able to recognize fully you know by doing a few more additions etcetera about the limit cases that is what happens when you know we have not said when the halting is happening and all that in this diagram, but all that we can take care of. We have not said that halting by saying yes the string is indeed like number of a is equal to number of b is equal to number of c, but if that is not the case what happens. So, all that we can incorporate that is not too difficult here, but already if the Turing machine can indeed solve this example successfully, then we already know that it can do something which no PDA could, because we know that this language is not a Turing machine. So, therefore, at least right now you are getting to believe that Turing machines can do things more than what PDA is ever could. So, can it do everything that a PDA could yes that is not too difficult, but right now let us not get into those details before we kind of appreciate and we the definition of Turing machine we are somewhat clear about working with Turing machines and then as the course proceeds I think you will be convinced that indeed Turing machines can code all algorithms. One more point before we end remember that this is for example, one Turing machine as of now there may be some more states later on will add, but right now it has a number of states number of transition. We get different Turing machines by of course, having different or set of states may be symbols may be transitions what we are finally interested in what all things the class of Turing machines can perform.