 Good morning and welcome to the fourth day of the workshop. We will look at the recursion in the morning session and in the session after tea, we will look at what we call the efficiency of our algorithm or the algorithmic complexity. There is one topic which is often not covered in the first course, but we do give a little bit of introduction, not necessarily from a theoretical point of view of algorithmic complexity as we understand it, but yes we do introduce some terms, but what we emphasize is that efficiency of our algorithms is of paramount importance and we try to substantiate it by some examples, so that is what we will do in the second session. So as I said we will be discussing recursion today, primarily we will look at implementing a solution through recursive function calls. To understand recursion we will revisit the normal execution of the function, we will see what happens when a function is invoked, what is the modus operandi that the program execution follows and in the context of that we shall see the recursive execution. We will also briefly comment on the computational cost of recursion, again traditionally when recursion is discussed in first year, nearly the mechanism of recursion is explained, but the implications of using recursion and the care that we need to take while deploying recursion is rarely discussed. I thought I will touch upon that because that is an important aspect, so you briefly look at iteration versus recursion. We have seen several problems in which we have evaluated functions where the algorithm was implemented using iteration, it appeared natural to us in Newton-Raphson method, it also appeared natural to us when we are finding let us say maximum of a set of given values or sum of n numbers etc. Even when we are calculating factorial of a given number it appeared natural to us. It was only in the Hemchanda numbers or Fibonacci series evaluation we encountered a first situation where the definition of the series itself was given in a with a recursive formula. The definition itself is recursive basically and we were trying to implement it through iteration which we did successfully and with reasonable clarity. However, there are problems where the recursive definition and the recursive implementation in fact is far more elegant and simple than the implementation using iteration. Although we are not going to discuss that problem in this session and many of you would be familiar with that, the problem that I mentioned is Tower of Hanoi problem. There is one problem where not only it has a elegant recursive solution but indeed implementing that particular solution using iterative techniques is rather clumsy which is not so in most other recursive situations. So let us look at problems whose solutions can be defined in a recursive manner. The most familiar to us is the factorial function f of n which is defined as n factorial equal to 1 into 2 into 3 into etcetera etcetera into n minus 1 into n. So it is essentially the product of successive integers of 2l that is factorial n. This can also be expressed as f of n is equal to n into f of n minus 1. Now this is a recursive definition. The nth term is being defined in terms of the function itself namely the n minus 1 term into n. You will appreciate that many series that we study in our schools etcetera actually render themselves for an elegant recursive definition many times. But let us concentrate on the factorial problem. First of all to understand recursion it is best to state a simplest definition of recursion namely a definition is something in terms of itself. In real life this could lead to a problem. You might have come across a circular definition where you ask somebody the meaning of a word and that somebody tells you the meaning and then after some time you ask you the meaning of the other word and you get back the first word in place. So this is truly recursive it is actually a circular definition we are not talking of that we are talking of mathematically elegant concept where very concise definition of operations or operators can be given. So coming back to the other problem which is the Fibonacci series or the Hemchandras series as I have called it the nth term is given by f of n equal to f of n minus 1 plus f of n minus 1. Notice that there appears to be a very small difference between this kind of definition and the definition of factorial. Factorial nth term was defined in terms of n minus 1th term Fibonacci series nth term is defined in terms of two previous terms f n minus 1 and f n minus 2. Both of these are recursive definitions as all of us understand. The question is can we implement our computations using recursive cause to functions we have seen functions are called we set up an iteration in the main program and within the iteration each time we may call a function that is the typical way in which we have exploited functions. What happens if we define a recursive way of computations? First of all we must understand that such recursive definitions are meaningful if and only if some conditions are prescribed firstly you need some initial conditions because you cannot kick off the computations. If nth term is defined in terms of n minus 1th term then you can go back and back and back but somewhere you have to start otherwise you will reach minus infinity. Typically then for n factorial you define f 0 equal to 1 this is the initial condition once one term is defined since each term is defined in terms of a previous term given f 0 I can calculate f 1 given f 1 I can calculate f 2 and so on and so on and so on. So I define this as f 0 equal to 1 and f n is equal to n into n minus 1 for n greater than 0. Now that is mandatory if you do not put this condition because the series is not really meaningful for negative n. Similarly the Fibonacci series terms are defined as f of 0 is 1 f of 1 is 1 and then you define f of n as equal to f of n minus 1 plus f of n minus 2 and we say this is valid for n greater than 1. So these are the two recursive definitions with which we will work to illustrate how recursive function calls can be used to implement similar solutions. Before we go to the recursive function invocation let us first get a little bit more information about how our normal calls to function are executed. So for example when we write a factorial function integer factorial integer n we start with integer f equal to 1 if n equals 0 we return f for i equal to 1 to n we then calculate f is equal to f into i. So this is the successive accumulation of the product when we complete this iteration n times we would have got the product of n numbers and that is stored in f so we simply say return f. So when we say return f the control goes back to the calling function. Please note that there is nothing like recursion or anything here it is plain function some parameter comes in the function starts executing itself and at the end of the execution the function returns the value which incidentally happens to be the value of factorial n and how do we invoke this function? We invoke this function in the main program we may say integer n equal to 3 this is just a sample case so effectively we are trying to evaluate factorial of 3. We define another variable called answer as an integer variable later on we will see that it is not essential to have this variable we can simply keep using the command return to return the appropriate value but nevertheless here we are saying that we will give the function a value 3 or value n and we will expect the return answer to be given in terms of answer. The expression that we write is answer is equal to factorial of n so this is just like scan f print f or any square root any library function this is our function we have written it right hand side is an expression the expression has only one term which is that invoke the function the function will be invoked the company will go over to that function calculate the function value and will come back with the return value that value is replace this term in the expression is the only term and the final value will be assigned to answer simple so the main program invocation is actually simple and straightforward answer is equal to factorial n and we print that answer what we wish to understand is what happens when a particular invocation is done so let us understand the sequence our computer will start executing instructions of our program just follow the arrow it will do something here most probably look at the definitions and whatever maybe there is an input statement here we are giving n equal to 3 specifically so it will look at execute this statement sorry it will allocate space for n as an integer number answer as an integer number then it will execute this statement while executing this statement it has to evaluate the expression while evaluating the expression it comes across a function reference it is precisely at this point that the computer will stop execution of this program well I should not say stop it will suspend execution of this program and because a reference is made to factorial it will sort of go over to a place where the code for that function is stored inside the same translated or compiled code is already there for the function there would be an entry point just like we have int main there will be an int factorial or whatever and the computer will go over to that function we want to examine what happens during this transfer of control and also after the function calculates the return value and the return value is brought back by the computer to this particular main program it is obvious that if the computer left computations at this point suspended the execution of this program and went over to the function then it must come back to this point as I mentioned yesterday the reference factorial of n will be replaced by the return value whatever it is since in this case n is equal to 3 3 into 2 is into 1 which is 6 will be the return value and that value will be assigned to us so what happens when the function call is made we have actually clearly understood the sequence of activities namely that whenever a function reference is made the computer hands over control to that function and when function completes the computations it comes back but there is something settle that happens without which actually this transfer of control transfer of value etcetera cannot be facilitated I have tried to show it through a diagram here first the activities that happen the control is handed over to the factorial function with the value of parameter n equal to 3 the computer now allocates memory to the variables used in the function for example to f i n etcetera what is important here is to note that this n which is part of the function definition is different from the n in the main program although n is used as a parameter by us here let us go back to the previous slide factorial n so we are sending this n as a parameter similarly go back to one more slide we will see that the factorial itself is defined in terms of int n but the fact that this name is n and the other name is n is merely incidental I could have said x here y there actually in fact these n represents a different memory location and the n in our program represents a different memory so logically you may imagine that the value of n is being copied by the computer and is being as parameter and is actually being assigned to the n which is the corresponding parameter in function so there is actually a transfer of value so apart from assigning memory locations etcetera etcetera n having a separate memory location the value needs to be transferred it is not only the value but something else also needs to be transferred what is that this is roughly the diagrammatic representation of the function invocation so this is the main program and this is the function so let us again recapitulate what happens the main program was executing the instruction answer equal to factorial n at that time it notice the reference to factorial and it decided to jump out so what it has to do it has to remember the instruction being executed please note that a function reference can occur anywhere in my program my program may have one thousand lines of course okay a function reference may happen in 37th line obviously the computer has to remember that it was executing 37th line so that when the function returns the value it must start exactly from the point where it left in fact it is not even the 37th line imagine it was a complicated expression and during within the expression somewhere there was a function reference that the computer has to remember the exact point where the function where the evaluation of the expression was suspended so remember the instruction being executed and save the context what is the context at this juncture our first year students will not understand the meaning of the context of program in technical sense suffice it to tell them that the context means that when the computer is going over because there is only one computer it is useful to emphasize that there is only one processor and at a time it can execute instructions only of one program so when it goes over to some other program when it comes back when it goes back goes over there are a whole lot of things which the computer knows when it is executing the program that is called the context in technical terms it could be open file handlers it could be register values whatever what typically the actual actually happens is that all such context information gets stored into some kind of a stack and the computer's processor is handed over control to execute the function the function execution requires the parameter values so computer faithfully collects all parameter values and then hands over control to the function so remember the simile that I gave main dumbo and assistant dumbo effectively it is the same computer but effectively the control is being handed here and when the control is handed over to the function variety of additional things happen I have used the term here set up a logical block for factorial execution there are already instructions there so what do we mean by setting up a logical block the computer actually looks at that function just as it has got rid of the context by saving it somewhere the function may have some other context you have to establish that context you have to actually allocate memory to those variables and then you have to calculate the answer after giving the parameter value that you have brought along with it so in the least if you want to simplify it and still be conceptually correct you can tell your students that look the minimum work that this computer has to do is to take the parameter values and and push them or assign them to the corresponding parameters here having done that the function will calculate the answer then after calculating the answer the allocated memory may have to be released I'm not talking about the memory which is allocated to static variables we have not discussed the class of storage yet but all of you know that we are talking about the memory which could be temporarily allocated while the function is being executed all that will have to be released and then the control will go back from the function to the main problem so every time a function is called this kind of sequence will happen and what is written inside this block here and what is written inside this block here is the equivalent of the overhead that you allow to incur in order to go over to the function and come back please note that if I had written the computation of factorial in my main program itself as far as calculations are concerned I would have carried out exactly the same number of calculations but there would have been no overhead if I did not have a function of course functions are useful because they isolate segregate and modularize my code but there is a little bit of overhead that we should be aware of ordinarily this overhead not not a worry however when it comes to implementation through recursion then we suddenly see that in some cases these overheads could simply flare up in a nutshell there are two types of overheads which accompany a function called first of all additional memory may have to be allotted allocated every time a function is invoked one for the data elements defined and used in the function and the other for temporarily storing the context of the current program and there could be an additional computational overhead in terms of transfer of control parameter transfer etc now let us look at the factorial function execution using recursion if I am executing factorial using recursion this is the way I would write the program I called it program 6 dash 1 point C calculate factorial of a given integer n and I am defining the function integer find factorial int n I deliberately change the name slightly to distinguish between a recursive evaluation and the conventional evaluation so again I use the same notation I define an integer value f but now look at my computations I am not solving it iteratively so this function does not implement computation of factorial by iterative method instead I am using the bare bone definition of the factorial itself which we saw just a few slides ago so what I say if n is equal to 0 I set f is equal to 1 however if f n is not equal to 0 I say f should now be calculated not by any computations but by calling the same function find factorial but with value n minus 1 multiplied by n you will recall this is the definition of factorial n factorial n nth term is equal to n minus 1 s term multiplied by n I am actually here to calculate nth term if n is 0 I know the value is 1 but obviously n is not 0 so n will be 3 4 5 10 whatever I will come to the else statement first then I say f is equal to find factorial n minus 1 star n the computer will suspend execution of this program this is not the main program the function itself but it will suspend its execution it will calculate the value of n minus 1 which is now the new parameter and with this parameter n minus 1 where will it go it will go back to itself in the process it will save the context of current execution which is again the function itself so it's a very complicated thing I am going to ask myself to do some computation but so that I should not forget what I was doing I will remember the context somewhere and then call myself so obviously there are two me's now the first me who was called by the main program now that first me has called me I am now the second me but the first me logically exists the first me has not gone away the what is the difference if the second me that is the second time invocation of the function is at this instruction because that is where the computer has come the first me has gone to sleep while evaluating this expression so the first function invocation is somewhere here the second function invocation is somewhere here when the second invocation starts the first one is pending second invocation starts you will again find that for the parameter value n equal to 3 the second invocation would have been invoked with parameter 2 and again n is not 0 here it will come back here again and it will say f is equal to find factorial of 1 now n minus 1 so the second execution will have to be suspended third execution will have to start let us let us describe this situation pictorially so we are looking at the main program calling the function factorial I will call it fact and what is the value this main program sends it since n is equal to 3 the value sent is 3 here while evaluating the factorial it again encounters fact itself so this time the factorial is to be called with value 2 so therefore there is another fact that is invoked but this time it is invoked from here with a value 2 when you come to the program execution of factorial with value 2 inside you will again execute f is equal to factorial n minus 1 where n minus 1 is now 2 minus 1 so now the calculations will say no I need one more factorial this time it will execute the factorial with a parameter 1 when you come with parameter 1 notice the code if you recall the code the code says if n is equal to 0 then you set the value to 1 otherwise you again invoke the function with factorial of n minus 1 since 1 is not 0 this fact will again try to do that computation and encounter factorial and again in hope another factor this time it will be in hope with a parameter value of 0 please note that this factorial is not doing anything this copy of the function let us say logically there are now four copies physically there may be only one copy instantly those of you who have studied more details of compilers and operating systems you might have heard the term what we call re-entrant code so that means you have a program that program can be entered again and again and again every time you enter it it will swap the context it will remember the previous context and start execution from the beginning logically it is equivalent to what logically it is equivalent to saying that I have a copy of the program the first copy which I started executing when I encountered myself I made another copy and started executing that from the beginning so in terms of suppose there are instructions like this one two three four five one two three four five one two three four five etc when the program executes the function when the function is executed it might execute first second third fourth instruction and it might sort of park here when another function reference is made the other copy starts with the first execution of the first instruction then this copy executes instructions it again encounters another reference it goes further down and so on all these copies are dormant and each one of them remembers at which point the execution was suspended why it remembers it because it will have to come back to that so it is not as if these recursive calls are made and the final invocation of fact with the parameter zero so let me write the parameter here this is three and this is two and this is one and then this is zero when fact is invoked with zero during the computation it now suddenly finds out that condition if n equal equal zero yes the parameter is zero the factorial function is happy no further cost to itself it will now say go back and now the value will be passed what value will be passed back the value which will be returned will be for the case when parameter was zero which is one because we had said if n equal equal zero return one f is equal to one this value one comes back here notice that the invocation has been done so that factorial value which has come back is now one now one into n what was the end here this was one one into one is still one so this also passes back a value one when it comes here I will I will like you to recall that effectively the function that the value f is being defined in terms of f of n minus one into n n was the parameter so at any point in time you imagine as if this this is valid so here when this function is completely evaluated you actually not reach this point because if n is zero you simply return one but when you come back here this f of n minus one will be replaced by this one it will be multiplied by n which was the parameter in this case and therefore this one will be used so one into one you will get one now this one will be multiplied by two you will get two then these two will replace that function value it will get multiplied by three and this will become six and this is how you will come back here so the final value which is returned to the calling program is six so to recapitulate every time this function is invoked this function will either at the end will evaluate f as one and return otherwise it will come across this it will call itself every time it calls itself it will calculate return with some value notice that actual value is returned only when it is called as many times as is required to go to the bottom of the problem namely to reach n is equal to zero so let us say n is equal to 20 if n is equal to 20 this function will invoke itself 20 times and it will return value exactly in the same sequence going back 20 times consequently you can easily see that the cost of actual computation namely doing a multiplication is a trivially small as compared to the overheads which will have to be incurred in passing control passing parameter collecting parameter and so on ordinarily we would not be worried because the computer is a very fast machine so it is perfectly fine that let the computer do the computing but in some cases the function invocation in a recursive form is not as simple as this this is like almost creation of a tree with several nodes but in a so this is almost like a tree but with a single branch okay there are no branches here you are it's a linear tree coming here going here going here going here what if you had a function which called itself twice in the same invocation with different parameters that is precisely what happens in case of a Fibonacci series let's go back to the Fibonacci series definition look at this definition the function can be written in a recursive fashion but the way it will have to be written is some value f or x or y whatever the value to be returned will be defined in terms of invocation of that function itself with n minus 1 as parameter plus invocation of that function itself with the parameter n minus 2 can you guess what would happen it would perhaps increase the total number of invocations by a factor of 2 ordinarily I would have asked you a quiz on this but I have not prepared a quiz those of you who think that it is only a double increase in the invocation you are very very wrong the increase actually is exponential although it is not decipherable from this deceptive definition so let us try to draw a similar picture what we had drawn there for the case when we are evaluating in exactly the same fashion as we evaluated factorial by writing the recursive definition we write a recursive definition for Fibonacci series and what would happen when that function is invoked it is an interesting case to draw this time you will see that really you get a tree so here is my main program first of all how would I write my function invocation let me again get back to this old page the Fibonacci number let's assume that we are evaluating some variable f this will be evaluated if it is nth number that I am evaluating and the Fibonacci number is defined in terms of f of n minus 1 plus f of n minus 2 and of course there are logical conditions if n is equal to 0 if n is equal to 1 the values are defined as per the initial conditions we will not look at it but these are the initial conditions now with this kind of function definition if I start executing when my main program first comes let's imagine that the main program is trying to calculate the 5th term of Fibonacci series so the main program will come with a parameter n which will be equal to 5 this will come to the Fibonacci function so let me call it the function 5 now when this Fibonacci function is being executed it will have to calculate that f which will invoke Fib again but with a parameter value 4 it will have to come here this Fib4 will call Fib again and this Fib will have to be evaluating f is equal to Fib3 now plus something this is plus Fib3 notice that the computer has not yet looked at this Fib3 it is evaluating this expression and while evaluating this expression it is just concentrating on evaluating this term while evaluating this term it has to call this function itself it gets called and now for evaluating this this will have to call Fib3 and this will have to call Fib2 again Fib2 will be temporarily forgotten it will go and calculate Fib3 when it calculates Fib3 again the Fib3 is defined in terms of Fib2 plus Fib1 if F0 and F1 are only defined then this will still calculate call something else here when this computation now returns it will actually replace this value with whatever it has got and now it will go to this expression this part of the expression it says I have to evaluate F1 so it will again spark off another execution with all these Jamela when it goes back it will take one value back here but when it comes back it will only replace this and when it replaces this it will look at the remainder of the expression in the remainder of the expression it says F2 so it will again spawn off another call to Fib with F2 please note that when the computer comes back here it does not remember that it has already evaluated F of 2 that is what it did when it went there for computer this is merely another term in the expression it incidentally happens that this term requires invocation of function with the parameter value 2 recursive invocations do not retain the memory of past references and therefore it will end up making this call which again in turn will end up making another call all of this goes back here then the story will start all over again because the computer will say now I have found out the value for this term after all the Jamela now let me evaluate remainder part of the expression but this expression says 53 I will start again invocation of Fib with 3 as the parameter and again this and again this etc. So what do we have here what we have is a full-fledged tree multi branch tree in any tree the number of leaf level nodes at a given height are in what proportion they are not double or triple of N okay they increase exponential consequently this computation could become extremely costly for even small values of N I am not meaning small five or six or seven what I have done for your benefit is although that's the matter of discussion of the next lecture namely the algorithmic efficiency my colleague miss nagesh karmari wrote a program for this and executed it on Ubuntu and executed it for different values of N he did it using iteration of the iterative computation of February series and he also did it using the recursive computation some very funny things you may observe for example when you recursively evaluate factorial the overheads are not much but the factorial value itself becomes very large and when that becomes very large computationally you start getting errors however he and I decided last slide that since so many computations are being performed even if the value is wrong if we just want to find out how long it takes let it carry out those computations wrongly so let me get wrong results but let me see how long it takes and you can actually see the exponential behavior of that algorithm when you call it recursively by giving larger parameter values that of course is a matter of discussion of the subsequent class post t