 Hi, and welcome to this next lecture on Data Structures and Algorithms. In today's session, we will discuss the concept of running time of a program. More specifically, we will introduce you to how you could analyze an algorithm based on its running time. We will consider two notions of running time. One is empirical and that will lead us to the analytical method. So, let us take a canonical example and that of search. We will illustrate through, we will illustrate the canonical example of searching element E in an array S, you know E could be anywhere in this. We will make an assumption that S is sorted and we will assume that the sorting is based on the element, the key value of the elements. So, the running time will really depend on how long this sequence S is. Also, the running time could depend on the position of the example. So, how do you characterize this running time as a function of input size and the position of the element? So, to empirically find the running time, you need to do the following. One, you need to run the program and this is based on the choice of the algorithm. We will discuss two algorithms for finding such an element. Now, you will need to run the program corresponding to that algorithm not once, but multiple times and that too with a large number of different kinds of inputs. What is the difference in the inputs? Well, with change in the input size and position of E, you will have different inputs. So, these two are the different input parameters we could consider. For every run, you will need to compute the time to run, the wall clock time and also compute average times across runs. Now, moment I say wall clock time, you should be bothered about other intervening factors. So, to do the empirical analysis of running time, need to eliminate or more generally normalize all extraneous factors. Now, what are the extraneous factors? You could have background processes running that affect the wall clock time. So, you should have no background process running. The other thing is, if you have different programs that you want to compare, you should be talking about comparable data structures or comparable data types. What does this mean? Well, we will consider int and float to be comparable types. However, an int array is certainly not comparable to either of these. So, we need comparable data types. We will also assume that the implementation is single threaded and we will also assume that there are no idle time because of some scheduling problems, single threaded and no wasted cycles. Finally, you might want to compare algorithms, implementations of different algorithms. So, certainly you want to ensure that these algorithms are implemented on programs and that are in comparable languages and these programs are run on operating systems that are comparable. Even after all of this, you would not know what kind of system generated interrupts or any other extraneous factors related to temperature and so on might affect the running time of your program. So, you need to also average across multiple runs for each program. One good practice is to run virtual machines which have guaranteed resources. Obviously, you need to run these virtual machines on very high capacity servers. So, you profile the running times of different implementations and plot them one against the other. You could make some observations of the time complexity. So, we have taken this example of searching over this array. The binary search basically operates on sorted arrays, assumes a sorted array and as you increase the array size and profile the time required in certain units such as microseconds as a function of increasing array size, you find that an algorithm such as binary search grows very slowly with the size of the array whereas, linear search as the name says grows linearly. Now, in this simple case, is there a way we can talk of one algorithm being always faster than the other? So, notice that there are regions lower down here for which you might expect the linear search to be faster than binary search but how do you know that? Should be only based on empiricism or experiments. You could also plot the time required as a function of the position of element being searched for and this, you do not expect to be very informative but let us take a short nevertheless. So, the element E in the case of linear search as you increase a position, its position of E, this is the time. You would expect the time to grow linearly assuming a left to right scan with the position. How about binary search? Well, with binary search, if the element happens to be exactly in the middle, yes, certainly you take very less time. You would then invoke binary search on each end. So, there would be some time, some more time required depending on where the element is expected to be. Of course, the time would again grow as you move to either side and so on. So, depending on the resolution, then length of this list, you would expect a very zig zack and so on. I would not bother to complete it. This is something you can try. Now, if you treated the position of the element E as a variable, is there any insight that you might get? Would it be a good idea to average the performance of an algorithm across multiple positions of the element? Certainly, yes. You want to average, but there is nothing specific you will get by varying the time as a function of the position. There are some issues with such empirical analysis. The primary issue is that instead of reasoning about the outcome before the experiment, we seem to be reasoning after experimentation. Is there nothing that we expect from a particular algorithm? In fact, this whole method is too time consuming, reasoning, post hoc reasoning is very time consuming because there is a limit to the number of algorithms, there are a number of runs of an implementation that you can ever make. Also, there are several hidden factors despite our best efforts to normalize them, for example, the hardware, the compiler, etc. Most importantly, is that it ignores the essential behavior of the algorithm. Bind research is supposed to be efficient on a sorted array, but what does efficient mean? So, you want to capture the essential behavior of the algorithm. However, empirical analysis might ignore this while focusing on the unnecessary details. And there is actually no substitute to analysis on paper because you get a foundational understanding of any system based on pen and paper work. There is however a caveat to such analysis. You need to do serious modeling of the system. That is what we will cover in this lecture and some following lectures. So, our model for algorithm analysis is as follows. We are going to assume that we have a processor that sequentially processes instructions and that we have infinite memory to hold any auxiliary data. All basic instructions take one unit of time. This may not be the case in practice. We assume that addition, subtraction, multiplication, division, bit operations, comparison, assignment, etc. all take same time. We know from practice that multiplication and division could be lot more expensive than addition and subtraction. However, you could multiply a number x by 2 by basically shifting, bit shifting x to left by one bit position. In fact, you can do the same thing for kx. Kx is shifting x to left by k positions. So, we will assume that these are all to be treated on the same scale. And we will make these assumptions for both integer and floating point data types. So, in this process, we are ignoring discrete times, memory hierarchies, paging, context switching and so on. So, this is really moving away from our messy empirical setup. So, let us consider this algorithm A for searching element E in a sequence S. So, recall that we are first interested in a linear scan algorithm. So, this is a linear scan algorithm which takes as input the element E and the sequence S. And as the name says, it scans every element, checks if the element is the number that you are searching for, the num here is the E. If found, it breaks, else it continues the scan. So, in the fall loop, you are making one increment at a time. The cost is one. You are making array accesses and comparisons simultaneously. This overall cost for the comparison is two. And then both assignment and break statement will incur a cost of one. So, the overall cost will basically depend on where the element is found. So, in the case of successful search, you will keep doing these comparisons and the increments till the element is found. And if the element were found at the end of the array, you will have to do these up for n or n minus 1 steps. The assignment steps, the last two assignment steps will be called exactly ones. So, what exactly? So, summing up, the time for successful search will incur a cost of two. This is for the if statement where you make an array access as well as a comparison. This will be incurred n plus 1 times. You will also have the increment operating n plus 1 times. In addition, you have these costs associated with termination, the one and the one. So, overall, you will find the cost to be 4n plus 5. How about an unsuccessful search? Which means you actually had to go until the end of the sequence s. So, in such a case, you never come across the found and break statements. So, you will have to do this until the end to increment the array access and the comparison. So, the cost incurred here is basically 4n plus 2. So, please note the comparison here is between 4 times small n plus 5 and 4 times capital n plus 2. So, how you compute the average number of instructions executed by program a? And when we talk of average, we mean what? We mean average across all possible values of the position of num or the element e. While holding the length of the list as a constant, this is the kind of average we get interested in. And why is this so? Well, we already seen that looking at the time as a function of num is not really meaningful. However, looking at the time as a function of length is meaningful. So, you hold the length constant average across all values of num, change the length and average across all values of num and so on. And basically, we expect to see the empirical results but anticipate them analytically. And we would expect the linear curve to go up like this as we had seen already. And the logarithmic curve to move very slowly. You would like the mind research to operate like this. But we would like this to be evaluated or validated on a large number of possible sequences s with different positions of num in order to get the expected or the average performance of both algorithms. Thank you.