 So, while today the main topic of discussions is multidimensional arrays, we are going to first continue our discussion on salting. You remember the other day we looked at an algorithm which will take an array of numbers and sort them in ascending or descending order. And we gave an example of salting marks in an examination and we said how if we have an associated roll number array, then whenever you exchange any elements of the marks array, you must also exchange the corresponding element of the roll array so that the sorted arrays remain consistent with roll number and marks. Now, you will recall that we had commented on the execution time taken by our sort algorithms. And we observed that if we increase the number of elements to be sorted by let us say twice the previous number the execution time appeared to increase four times. It was essentially moving in proportion to the square of the size of the problem. At that time we merely noted that and we said that there has to be a faster algorithm for doing sorting. There are actually many fast algorithms which do sorting better. One of them is called the merge sort. To understand the notion of merge sort, we are not going to discuss the merge sort algorithm today. But we are going to discuss a key problem in merge sort which is if I have two sorted arrays, independently sorted arrays, then how best I can merge the elements of these two arrays into another third resulting array which still retains the overall ascending or descending order. Of course, that can be done but how can it be done in minimal amount of time and in general what is the time execution time that we will have to spend how many operations we will have to do. So, we will discuss that. After discussing that problem we will move over to the discussion of another important data structure which is available in C plus plus namely multidimensional arrays. We will specifically look at matrices because that is what we have to deal with in most of our computational problems in science and engineering. We will take an example of solving a system of simultaneous equations which is something which is commonly known to all of you. We will look at the mechanism of representing such a system of simultaneous equations using matrices and we will then look at how such a system can be solved by one of the well-known methods which is called Gauss elimination. You might not have studied Gauss elimination technique but you would have studied matrices, right? You are familiar with the matrix, matrix multiplication, etcetera, etcetera, matrix representation numbers, fine. So, this is roughly what we are going to discuss today. We begin our discussion with the problem of merging contents of two sorted arrays. So, here is a problem. I have an array T 1 which contains seven elements 15, 11, 9, 8, 6, 2, 1. They are sorted in descending order. There is another array T 2 which contains four elements 12, 5, 4, 3. Again T 2 is also sorted in descending order of values. It so incidentally happens that in these two arrays, the numbers that you see, the values that you see are actually unique values across both the arrays. It is not necessary. After all, there could have been 15, 11, 9, 9, 8, 6, 6, 2, etcetera. So, the same value can recur, just as in case of marks for students, there could be several students scoring the same marks and if that happens, then their relative position in an array does not count really. If somebody may be a third element, fourth element, fifth element, but as far as ranking is concerned, all of them will be considered to hold the same rank. So, as long as they remain sorted in that descending or ascending order, we call that array as a sorted array. Our ambition is that we want to construct an array R which will contain elements from both these arrays, but which itself will be in the descending order of element values. So, consider now the job of how will you do that? Imagine these are the arrays in which case the index of this element will be 0, index of this element will be 1, 2, 3, 4, 5 and 6. The index of these elements in array T2 will be only 0, 1, 2, 3. There are less elements. In general, consider that one array has say m elements totally and the other array has n elements. It is obvious that the resultant array R will have m plus n elements. What is the best way of getting the results into array R? If you want to scan any array T1 or T2, you would set up an iteration which will move some index value from 0 to m minus 1. This array can be spanned by moving an index from 0 to n minus 1. However, you will notice that you do not want to scan a complete array, because the values that you will have to put in the result R here cannot be decided by looking at all the elements of any one array. You have to actually look at elements in each of these arrays, keep comparing them. Whichever is the larger element you want to push into array R and then move to the next element of that array for comparison. In short, you need to set up an iteration which will actually move an index for array T1 and another index of array T2 in an asynchronous fashion. They will not either move from 0 to m minus 1 and 0 to n minus 1 one at a time, nor they will move 0 here, 0 here, 1 here, 1 here, 2 here, 2 here, because at any point in time you will notice that you will be comparing different elements of these two arrays to decide which one goes into R. Consequently we decide on an index variable let us say K. This variable K will denote the index for T1. Let us take another variable say L which will denote the index variable which will scan T2. We will start with K equal to 0 and L equal to 0. What will index R? It can be neither K nor L. We require another index expression because R index is going to vary from 0 to m plus n minus 1. There are going to be larger elements in R. So let us decide that for R we will have another element let us say J. J is also initially 0. Now what we are interested in to figure out what happens to K, L and J, what should happen to K, L and J and what should be the manner of pushing final result into array R. Consider this when K is 0 and L is 0. I am essentially pointing to T1 K will point here and T2 L will point here. I am comparing 15 and 12. The larger of the two should go first into the element R. So larger of the two is 15. That means 15 should get into the array R. It is its first position. The moment 15 goes here J should become 1 because next element is going to be element number 1. Should L change? L should not change because L has not moved yet. The element has not moved yet. What should change? However is K. K should not now point to the next element. So K should become 1. This is the progression then that if I have moved an array from T1 that means Kth element I have moved to the corresponding element of R then K should become 1 now. At any point in time I will keep comparing Kth element of T1 with Lth element of T2. So if K is equal to 1 now I compare 11 and 12. What do I find? Now 12 is greater. So obviously 12 should go into R and the array from which an element goes to R its index should be increased for next comparison. So I have here after 15 I will get 12 here and L will now become 1. In the next and of course J must also increase to 2. Now what am I comparing? I am comparing with 11 with 5 because both indexes are 1 and 1. Obviously 11 is larger. 11 will move here but index for this K will increase. Next time I will compare 9 and 5. 9 is larger 9 will move here. The index of K will again increase to 8. It will again increase to 6. So observe that if I continue like this the index for array T2 remains stagnant as long as I keep finding larger elements in array T1 unless all of them are exhausted and are put into R I am not going to change L. So what will I have here? After 15, 12 I will have 11 which is the element of array T1. So K will now become 2. K2 will be compared here 9 will be compared with 5. It is still greater so I will have 9 here. Of course this would have become 3. K will now become 3. I will look at value 8 here. 8 is again larger than 5. So I will push 8 here. This has of course changed to 4. I have got 0, 1, 2, 3, 4 elements here. The next value actually is 5. So J would actually have been set to 5 ready to receive a number. Now T1 3 which is 8 has been compared. So K should increase now to 4. So you will notice that K and L are moving as synchronous. The moment K is equal to 4 and L is equal to 1 I am actually comparing the fourth element here with the element number 1 here which is still 5. So that means I still have to push 6 here. When I push 6 here this will increase to 6 and this will increase to 5. Now I look at the fifth element which is 2. When I compare these two with 5 the index here is stagnant at this point. Now I notice that 5 is larger than 2. So this time I won't push the result from T1 but I will push into result from T2. And this time 5 will come out here. This of course will increase but this time L will increase to 2. Notice that after 5 the next element is 4 which is larger than 2. So again you will get 4 here. This will increase to 3. Then again 3 is larger than 2. So you will get 3 here. I am sorry this will remain. You will get 3 here and this will become 4. Is there a fourth element in T2? L has become 4 because I have already compared the third element. I found the third element 3 to be larger than this element 2 and I have pushed these 3 out here. Technically I should increase this and at any point I have to compare Kth element of T1 and Lth element of T2. Kth element of T1 is well defined. It is still 2. What is the Lth element of T2? An element does not exist. Now if I write a general algorithm which will continue this till I have exhausted all m plus n elements I will get into a problem here. Depending upon what arbitrary value happens to be in the memory location corresponding to the next element of this array which is non-existent that will get compared. The correct strategy to be used in such cases is called a strategy based on putting a sentinel value. I think I had used this term once. Sentinel is like a sentry or a watchman which you keep on the door. The watchman decides who is permitted to go in and who is not permitted to go in or there could be an at the exit point. This time we are talking about putting a sentinel value at the exit point such that our logical comparisons can continue. Imagine if I had some element value here which was guaranteed to be smaller than any possible element anywhere. It is an artificial value which I will insert. If I did that then the comparison with the fourth element will proceed normally because there will be an element with index four here. Such an element which is artificially inserted typically at the end of an array to facilitate continued comparison with the index expressions that we count are called sentinel elements. In this particular case when we are talking about marks in an examination and we generally presume that marks are not negative in which case any negative number such as minus 999 minus 999 arbitrary number. Suppose after reading the sorted array I inserted minus 999 myself as an artificial number in the next available element position. Now do you see that the proceedings will continue in a normal fashion because right now I have k is equal to 5, l is equal to 4. The element at index 5 is 2. The element at index 4 is now minus 999 and it so happens that 2 is greater than minus 999. In fact that is why we have designed this value space. So this comparison it will say this is larger than what will happen now. So we will get into the result. So I get 2 here and I increase the value of k to 6. I take the sixth element. Sixth element is 1 which is also larger than minus 999. So I will get 1 here. This will become 7. But by this time the index j which we have completely forgotten would have increased every time I insert something into this array. So this would have become what? Yes. 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 itself means 11 elements have gone in. 11 is the index. The moment I reach 11 index I know already that I have exhausted all the valid elements of T1 and T2 because there were 7 elements here and 4 elements here. That is a terminal condition I get. I need not carry on beyond a value of j which is equal to or greater than m plus l because there is nothing to be compared. Doesn't this indicate a simple logic then? I scan the two arrays as synchronously associating one index with one array, another index with another array. Keep moving comparing the corresponding elements indicated by those index values. At any time I find a value larger I push it into the result and move the index one more. I keep doing this as synchronously for both the arrays and I keep repeating it. Every time I put an element into result R and I will put one element in result R in every iteration, one of the larger. And whenever I have finished m plus l elements I will just get out. The total time and effort that I will spend will be proportion to m plus l because I am executing an iteration only m plus n times. There are no nested iterations here like what we saw in sorted array. There is only one thing. I must be able to find a sentinel value. We are discussing now situations where the actual values in array are defined over certain domain in which case any value which is legal but outside that domain can be taken to be sentinel. And depending upon whether we are finding out maximum or minimum the last value can be appropriately adjusted. However if the arrays contain all possible legal values then it is not possible for us to consider an artificial sentinel value. We shall see what is to be done in such cases. We shall see what is to be done in such cases. However for the purposes of illustration and for most of the practical problems this sentinel strategy will work. It so happened that in this case the last element of array T2 happened to be larger than one or more last elements of array T1 and therefore we needed a sentinel here. But we do not know the actual data values. It could be any which way. The best way then is that read both the arrays and push a sentinel value as the last plus one element in both the array. And then it does not matter whichever is encountered first you will actually do the computations correct. So is this process understood as a very simple technique in a single iteration which is executed m plus n times you compare corresponding elements of these two arrays and decide which one to push. So this is a program to merge two sorted arrays. The program says that input is to array T1 and T2. T1 has m elements, T2 has n elements and arrays are of course assumed to be sorted. The result array are which will contain sorted elements. That is roughly what the program is supposed to do. Let us look at the declarations. I am again arbitrarily assuming that there are 100 elements in T100 elements in T2. It could be thousand. It could be million. As a matter of fact the extension which I am going to ask you to work out you will have probably millions of elements in the array. M and n are maximum numbers. I am arbitrarily assuming p to be the maximum number of elements in R which will be simply m plus n. Arrays hold sorted numbers in descending order. This observation I make it here. This should have been made here or there because that will decide your comparison. And of course I have integer variables k, l, m, i, j, etc. which I will use for either reading the arrays in or for my logic of the main iteration. This entire slide is about reading the input properly. So I ask user to give number of elements in first array. I read m. Then I ask him to give individual elements and I read T1i for i equal to 0 to m. So this kind of iteration is very common by the counting iteration for i equal to 0 to m minus 1. Notice it is i less than m. If you put i less than equal to m you will get one extra number unnecessarily. So you put this iteration and generally if there is one instruction you can write it without brushes. But as I have said it is good practice to write brushes here because you are reading the value and you are also outputting that. Notice that this will be done iteratively. You will be putting those values one after another. And if you just put a c out statement T1i then those values will get covered up and you will not be able to distinctly read those values on the output. Invariably then when you are putting out element values in an iteration it is good practice to put that element value and put one blank space or two blank spaces after that. So that the next element when it is printed it is printed two spaces away and whatever. Just pretty printing. Notice that after having read elements from 0 to m minus 1 I am actually pushing a sentinel value in mth element. I am calling it as I put an end of line and continue doing exactly the same word for reading values of elements of array T2. So you can see a lot of instructions here to write all of them amount to a simple thing. You are collecting the data from the user and putting it in appropriate arrays. Now the main algorithm interestingly the main algorithm actually is very short. It implements what we just discussed the process that we just discussed. So let us analyze this. You remember the logic that we used we start with values of k, l and j as 0, 0, 0. I am using a do while iteration. The do while iteration forces at least one iteration and then it will examine the condition at the end. If the condition is true it will continue iterating as long as the condition is true. That is why I says do this that means keep doing this while this condition is true. Notice the condition first. Do this again and again how long while j is less than or equal to m plus l. It is obvious to you why this is the condition because once I have collected m plus n elements there is nothing more to be done. There are m elements in T1 and n elements in T2. What am I doing inside? Simple exactly the same logic that we used while looking at the explanation of the algorithm. I compare T1 k and T2 l. If T1 k is greater than T2 l then T1 k elements should go into the results. That is what I am doing. I am assigning rj to T1 k and I am increasing k. If on the other hand this is not so then T2 l must go into rj and l is incremented. So if this then push from T1 else push from T2 but in either case notice that this events here in either case increment j because after having assigned one element to r next time when I come back I must assign the element to the next position of r. The most important point being made in this extremely simple and straightforward algorithm is that the index values for your arrays need not always move synchronously or need not always complete a iteration from 0 to m minus that is what you have done so far in all your iteration setting. This is an example where you are scanning three arrays actually. Two arrays you are scanning for looking at the elements and deciding on which element to push into r you are scanning effectively the third array r and each of the index value is moving is a pointer which is moving at different elements at different points. But I hope you will agree that this algorithm is quite simple and elegant and this will achieve the purpose so at the end I have here some output now the resulting array contains how many elements 0 to m plus n minus 1 that is why you start the iteration from j equal to 0 end it for j less than m plus n and you just output rj with some blank so you will get the series of elements in r. Here is the sample execution of this merge program so I compile it with c plus plus you remember the input redirection that we have discussed instead of typing the values of elements of array every time I run a program I have put the array elements of the two arrays including the number of elements in a text file which is called sorted underscore arrays dot text and actually the program is reading automatically from that file however the program will religiously execute c in and c out statements c out statements will come out on the screen c in statements will read from that file so what you will see here is still the net effect as if you have typed the data from your keyboard no difference so here are the two arrays the first array which has seven elements the second array which has four elements and the resulting array contains this this is exactly what we expect now the point this is an illustration which works for seven and four elements there any reason to assume it will not work for six hundred and six hundred elements the logic is exactly same there any reason to assume why it will not work for ten thousand ten thousand elements or one million one million elements now we come back I will make some comment on the notion of merge sort the sorting algorithm that we had discussed if you remember we had executed that program for let's say twenty five thousand students fifty thousand students and one lakh students by artificially generating the data and we had observed that the execution time was increasing in proportion to the square of the size of the problem suppose I have n students instead of sorting the data for n students in a single algorithm I divided those n students into two arrays there marks I mean I am doing artificial data generation so I could do the following I could define such two arrays marks one something two something or whatever T1 and T2 and I could when I am artificially generating data for let's say one lakh students I will put mark one here mark two there mark three here four there five there six there etc I can arbitrarily distribute data in any which way into these two now I can apply that sort algorithm to work on half the data if my data is half that time will be one fourth I got now two arrays sorted and if I now merge them the merging is not proportional to nested iteration timing it is a single linear you will agree them that breaking a large array into two parts sorting them individually and then merging them like this is likely to be much faster in the next class I will show an example of that but extend the same logic further if I have million records to sort why break it only in five lakhs and five lakhs I can break it into four different arrays I can break it into eight different arrays the general algorithm is called K wave merge it is not a very simple logic because to write multiple arrays and to maintain those multiple arrays will make your program clumsy later on much later in the class we will see that the data will not be neatly available in your memory in fact much of the larger data is available in this files and sorting is most pertinent when it is done on large data residing on your disk files which cannot be accommodated in the memory most of the real life sorting and searching is done on data which is located on the disk files not in the memory of course you need to bring that data into memory to process anything you will be reading the data from the file just as you are doing it now we shall study those algorithms post mid-sem but you agree that the simple technique of merging is a fairly powerful technique and if used in conjunction with the sorting that we have studied it could yield significantly faster results for sort the general name for such algorithms which makes merging and sorting is the merge sort algorithm there are many types of algorithms some of them will look at later but most of them would be studied as a specialized branch of algorithms mostly computer science students study them but all others who do a whole lot of computations involving large data might want to look at those as well ok with this we now go over to the discussion on multi-dimensional arrays just as we have declared one-dimensional array and used it we can actually declare a two-dimensional array and use it or three-dimensional array because it two-dimensional array is something which is quite familiar to us which is what we call a matrix the way an array is declared for two dimensions is you simply add one more pair of square brackets and put the size of the second dimension here so just as a 50 would have meant an array of 50 elements a 50 40 means an array a which logically has 50 rows and 40 columns so you have 2000 elements declared there naturally you would like to know how do you access individual elements because those are the ones which will participate in any operation in any expression in any assignment so each element is accessed by reference requiring two indexes if you have a five-dimensional array you will require five indexes each index is written in a separate bracket for example a r j it means first a dimension is indexed by r second dimension is indexed by j there is no harm you can say a i i if you did that and if you imagine the two-dimensional array to be a matrix a i i will denote what the diagonal element so you have a logical reference frame in your mind because a matrix you can sort of imagine before your eyes and you get a two-dimensional array to be exactly same as matrix number of dimensions in t plus plus are not limited they are limited only by the total memory that you have for allocation in general you would not require more than three or four-dimensional arrays in most of the complex computational problems as well note that the row index i can have a value from 0 to 49 if the size is declared as 50 because the array index always begins with 0 and ends at one less than the size also that like in the dimension array the size of the array in the original declaration must be a constant value it cannot be an expression you can't say array m n there is no such meaning because m and n are variables whose values are not known to the compiler you will read m and n later in your program maybe but compiler has to allocate memory locations to that array when it is compiling not when the program is executed so variable values or expressions cannot be used to declare sizes of the art much later we shall see how memory can be allocated dynamically while the program is executed but as of now we'll just go by the rules of array declarations arrays must be declared with absolute sizes declared at the time of their appearance in the program and all rules for index expression apply for index of each time namely any index can be a complex expression i plus 4 into j minus x plus y whatever whatever the entire expression is evaluated values converted to an integer and it is that integer value which is used if that integer value lies within the limits that you have prescribed such as 0 to 49 and 0 to 39 here then of course you are accessing a meaningful element of that two-dimensional array if that value lies outside you don't know what will happen the results are unpredictable c plus plus takes no responsibility for handling such equip with this equip with the multi-dimensional array structure which is very similar to what we have seen here we can note a couple of points immediately when we see this if i were to scan all elements of a single-dimensional array which had let's say m elements i would run an iteration for i equal to 0 i less than m i plus plus and excess each element in sequence but if i want to access all the elements of a two-dimensional array i will require a nested iteration for i equal to 0 i less than m i plus plus inside for j equal to 0 j less than n j plus plus and inside that i will do whatever i want to do with a i j and that will be covering all the elements imagine the first dimension to represent roles second dimension to represent columns and what you have is a matrix representation of your data we shall use that matrix representation to look at a problem which is quite important in most scientific and engineering computations namely that of solving simultaneous equations these arise in many cases consider these two equations 2x plus 3y equal to 8 and 4x plus 3y equal to y you will agree that these equations in x and y in a normal plane will represent two lines the intersection of those two lines will actually be the solution of these two equations that is one case where you want to solve equations when you go further deeper into the computational problems in say mechanical engineering, aerospace engineering, civil engineering, chemical engineering where chemical engineering where you are simulating a large plant with large number of different elements doing different processing you will have to represent or model real life situations by an extraordinary large number of simultaneous equations or later even simultaneous differential and partial equations there is no way you could solve them analytically you will have to solve them numerical one method that we are going to look at attempts to solve such simultaneous equations numerically so we will just revisit this example is well known everybody knows exactly how you solve such problems all of you have solved a 2x2 set of equations sometime or the other manually ok how many of you have solved a set of four equations in four variables one two how many have done that for ten equations in ten variables nobody and we are talking about engineering and scientific computations involving hundred by hundred equations the point I am trying to make is there is no way some of these problems could even be attempted without some computational tool however since all of you have done a 2x2 solution let us very quickly revisit the process that you use you effectively do what you add these equations you multiply some equation by something subtracted from something else effectively you want to eliminate one of the two variables get the value of the second variable put that back into the first equation the other equation and find out the value of the second what I want to do now is to go through the process rather methodically citing algebra rules which are to be deployed in order to provide that process so first we will look at the matrix representation of these equations all that we do is we take up coefficients of unknown variables x and y two four four three put them in a matrix these are unknown variables x and y so I put them here I am sorry yeah x and y here and then this is the right hand side eight in one are you familiar with this representation of a 2x2 equation like this you can easily make sense because this in a matrix sense means two times x plus four times y equal to eight and four times x plus three times y equal to one very simple what do we do with this representation now first we look at what do we do with those equations we want to solve those equations the method that we actually follow depends upon certain standard facts which govern the set of simultaneous equations if you call that a system then the system is not affected if an equation is multiplied by a constant so this is actually a hard provable theory that if you take an equation multiplied by a constant on both left and right hand side it does not constitute new information in that system it is still it is unaffected the system is unaffected so suppose I do that and I multiply this first equation by point five I am getting here one x plus two y equal to four I call it equation one dash is this point five an arbitrary value it appears to be arbitrary value but if we notice what is the effect in equation one the coefficient of x has become one so we are trying to get on the diagonal elements a coefficient one that is the objective as we shall see in the more general case now what have we multiplied this equation with we have multiplied this equation with some value which depends on the coefficient of the first element this was two so we multiplied by point five equally divided by two that's all and we got this equation so this we have utilized one fact there is another fact of course when I replant these equations in the matrix form I now get one two four three as my matrix here and simultaneously I get four one as my right hand side array so notice that by applying this logic I am changing the contents of this matrix and I am simultaneously changing the contents of the right hand side array x y of course is unknown so I don't do any processing there the original coefficients were two four four three and the new coefficients are one two four three now I apply the second important fact namely if an equation is replaced by a linear combination of itself and any other row again the system is not affected now I do not know in which way you have learned this but this is a basic factor of algebra in terms of simultaneous equations in multiple unknowns in fact when you want to eliminate x or y in two equations what you do you multiply one equation by some value and subtract or add it to another equation what you are doing is you are generating a linear combination of these two equations and the fact that is important why you get right solutions is because a linear combination of any two equations does not change the system the system remains same so this is an important fact here and using that we replace the second equation which was four x plus three y equal to one so we want to replace equation two by subtracting from it four times the first equation what is the reason this has a coefficient four there is a coefficient one if I multiply this by four times and then subtract it from here the coefficient of x will be zero that means x will be eliminated in the resulting equation that is the process that we use for solving if I did that then from these coefficients which was equation one dash and equation two I multiply equation one dash by four and I get this one two zero minus five four minus fifty notice that what was one becomes minus fifteen now and what was three becomes minus five now this happens because of the linear combination that I make most important the coefficient of x is zero such matrices are useful because I have eliminated one of the variables here I still haven't got the final solution I will go one step further I will further multiply the second equation equation two dash by minus point two where do I get this minus point two from notice that the coefficient of y here is minus five can you see this minus five now I want to find out the value of y ideally I would like the coefficient of y to be one to make it one which is minus five if I multiply it by minus point two I will get a coefficient one y now you will notice that the matrix which I started with which was a proper two by two matrix is now reduced to a form where I have one on the main diagonal and zero in the lower part non-zero values in the upper part such matrices are called upper triangular matrices and one of the great advantages one of the great advantage of upper triangular matrices is they actually yield the results when put in the context of simultaneous equations from the last equation directly get the value of y using that value of y and substituting it in the previous expression I will get the value of x and I can repeat this process any number of times I can have hundreds of variables it does not matter so starting from the last equation and using back substitution we get the solution for all variables well what we have discussed is nothing but the Gauss elimination technique so in short the Gauss elimination technique uses certain standard facts of algebra two facts actually one is you can multiply an equation by any constant and second is if you make a linear combination of two equations that means multiply one equation by some value and subtract it from another equation the system of linear equations remains same and we use these facts to to successively do the following one we get the diagonal element of the first row that is 1, 0, 0 to be 1 and then below that we want to assure that everything is 0 we do the same thing for all diagonal if we succeed in doing that we can apply this back substitution and get results this was a 2 by 2 example but it could be 100 by 100 matrix the same logic will apply so this is the essence I want to get an upper triangular matrix with all elements of diagonal as 1 and then use back substitution the entire Gauss elimination technique can be written in just these three lines that's the essence now we want to write a program to do that we will not notice it in the small examples that we will try but in real life the coefficients will be all floating point numbers by the way they are not nice integer values they are complex I mean they are not complex they are floating point numbers where the precision matters when you do this kind of processing multiply a row by some value subtract the corresponding elements between two rows and so on you are susceptible to round off errors and the errors may blow out of proportion when you have a very large system of equations that is practical fact we are not able to notice that now but having noticed that the scientists who worked on the numerical analysis algorithms evolved many other alternative stronger processes some of the names I have mentioned here for example Gauss Jordan elimination rather than just Gauss elimination you have a slightly modified elimination technique there is a notion of pivoting there is a notion of partial pivoting there is another method called LU decomposition and in fact a large number of such methods the simultaneous equations which arise in real life are not very straightforward like this 2 by 2 equations that we saw you may have an over determined set of equations have you heard of over determined set of equations you have 50 unknown variables but you have 100 equations which of these 100 do you choose will decide how accurate your final result would be you might have an under determined set of equations you have 20 variables but only 18 independent equations somebody gives you 25 equations and say choose any but you suddenly find that some of those equations are already some linear combination of some other equations then they are useless they don't represent any new independent information so in real life there are lots of complications there we are not going to discuss those our ambition right now is to discuss the programming of Gauss elimination technique how do we write a program to get the Gauss elimination this is a useful reference I think I had mentioned this once numerical recipes an extremely powerful book anybody who has to do with numerical analysis in the future course of action in your own field should actually own one book one the first book was published called numerical recipes in FORTRAN many of you would have just heard of FORTRAN FORTRAN was the first higher level language ever defined in the world of computers it was designed in 1956 by a group of 6 people 3 from IBM and 3 for the users of IBM IBM is a well known company you have heard the name I guess in 50s and 60s all the computers ever manufactured in the world half of them were manufactured by IBM so IBM was the strongest player in the field of computers then and lot of important and useful research has come out of IBM initiative FORTRAN was one higher level language which came out of that of course later on when C and C++ became popular for computational purposes the same authors written a book in numerical recipes in C and now they have written another book called numerical recipes in C++ and when I say now it is quite some few years ago it's an extremely interesting book good book and it discusses a whole lot of problems including the problems of errors that may crop up due to the floating point round off and such things that happen this is the general representation of n variables so if I have linear equations in n variables I could discuss I could describe them like this n by n matrix a set of n independent variables x1, x2, xn and right hand side b1 to bn we note however that this may be the mathematical notation that we follow but in C++ unfortunately our indexes do not start with 1 they start with 0 and they go up to n minus 1 so our representation we may be better off if we write it like this a0, 0, a0, 1, a0, 2, a0, n minus 1 actually we will use this representation the independent variables will use the notation 0, 1, 2, 3, 4 there are no values here this will be actually the result that we will get but we will have all the values of the coefficients here and all the values of the right hand side using these two n by n array and another n element one dimensional array we need to find out values of the independent variables this is the general representation of the result that we wish to achieve after applying Gauss elimination as I mentioned Gauss elimination gives us the coefficients matrix in an upper triangular form all elements on the diagonal are 1 all elements below this are 0 and there are of course non-zero elements here and the method of finally finding the solution is back substitution so I will find xn minus 1 first substituting that in the previous expression I will find xn minus 2, xn minus 3 till I find x0 what is the process of getting this in the upper triangular form we generalize the process that we saw for the 2 by 2 matrix it is clear that I need to handle all the n by n elements here so I will require a nested iteration it is best that I start with the first row so my top iteration will begin from I equal to say 0 to n minus 1 now when I am having one value of I let's say I is equal to 0 I will concentrate on this column for I equal to 0 the first thing that I do is I get this coefficient to be 1 note that at that point in time all these coefficients will be non-zero based on the input that you have so while I is 1 I will move J from 0 plus 1 to n minus going over this, this, this, this value and what will I do I will take the J equal to 1 row and ensure that by linear combination I make this into 0 and I change the corresponding values here I do the same thing for the right hand side for b1 while I is still 1 I will make J equal to 2 now and I will do the same thing with second row J equal to 3, J equal to 4, J equal to n minus 1 in short when my I is 1 I would have handled all the equations once by getting 1 in the major diagonal here for 0, 0 and by putting all of these as 0 now I gracefully go to I equal to 1 so I increment I that means from the 0th row I go to the first row I don't have to do anything here this is in the final form when I come here I do the same thing for the I comma I th element which is now 1 comma 1 element I make that one by appropriate multiplication here and eliminate all the coefficients below this so this time I will have to run my iteration for J equal to 2, 3, 4, 5, 6 to n minus 1 when I comes to and I come here I will have to do it for J equal to 3, 4, 5, 6, 7 so I have a nested iteration and the logic is very straight forward exactly the same thing I have to do the number of elements the number of rows that I look at for any one value of I will keep decreasing one by one till I go to the last row if this is the simple logic the implementation should also not be very complicated in general we represent the stream of n equations like this in terms of the coefficients that we are now talking about dimensional array and the one dimensional B matrix we will be representing our equations as a 0 plus a 0 1 x 1 plus a 0 n minus 1 x n minus 1 etcetera is equal to b 0 and we are saying this is the upper triangular form in which the coefficient of x 0 is 1 the coefficient of x 1 is 1 in the second equation and so on and the coefficient of x n minus 1 is 1 in the last this is the form which we have to get originally when I read the n by n matrix called A or coefficient matrix whatever I would have of course non-zero elements anywhere we go back to our example 2 x plus 4 y equal to 8 and 4 x plus 3 y equal to 1 and we notice that we represent x by x 0 and y by x 1 we will have this kind of coefficient matrix where the coefficients are 2 4 4 3 b 0 is 8 b 1 is 1 that is our representation and we can solve this problem getting the same values of x 0 and x 1 this is the program again I am sticking to my arbitrary number 100 I have an equation set which is 100 by 100 equations 0 to 99 0 to 99 this is the matrix B which is the right hand side x is unknown ok so I will have to find the results in the r a x I am declaring i j k as variables that I will use for indexing purposes n apparently is the size of the I am using additional variables which I am defining such as divisor you can guess where I will use that divisor to make the coefficient 1 I will have to take that coefficient value and divide all elements of that row by that value because when x i i becomes 1 all other element coefficients will also change so I will use that as a divisor where will I use a factor for any row where I want to eliminate a particular variable and make it 0 I have to multiply that by a factor and subtract it from the equation the factor will change every for every row so I have another variable called factor and of course I will have to calculate some of partially determined variable values when I do back substitution so I have this variable you will find that you can easily identify the objectives of these variables and r a elements in your program this again is the routine unnecessary work not unnecessary necessary work which you have to compulsorily do input and output consumes non-trivial efforts so I run a nested iteration for i equal to 0 to n minus 1 j equal to 0 to n minus 1 I read all elements of matrix of course I have first read n I read similarly matrix B which has 0 to n minus 1 element so n elements again this is the Gauss elimination so let us look at the Gauss elimination carefully as I told you I have to have an outer iteration so I start with i equal to 0 to n minus 1 now I am on the ith row it incidentally i is 0 at the beginning but it does not matter at any point in time I am at the ith row at the ith divide each row by the coefficients on the diagonal so divisor is mat a i comma i once I divide the entire row by mat i comma i the coefficient of x i comma i will be 1 so I do not need to do that division I set it explicitly to 1 there is a point here sometimes the computations that you get may not yield exact result so wherever you know what is the result like a 0 or 1 it is best to force it ok so I am setting that to 1 now I calculate all coefficients in that row so if I am on the ith row then the subsequent elements are i plus 1 to n minus 1 and each element I will divide by this divisor so whatever is the element in that row I am dividing by there and replacing that I have taken care of the ith row at this junction I have got 1 in the i comma ith element and I have appropriately changed all the other coefficients but I should not forget the right hand side fellow the right hand side element also must be divided by the same divisor so therefore I normalize the corresponding RHS element by dividing ith element of B by the same divisor so this part of the program has taken care of ith row of my Gauss elimination process now I have to take care of all rows below ith row and to ensure that I get 0s below that I comma i element this is done by this part where it says that we replace subsequent rows by subtracting the appropriate portion of i patient from that row and what is the appropriate portion whatever is the coefficient of x i in that row because the coefficient of i i is 1 I have set it to 1 so what is the coefficient there that is the multiplying factor I multiply the ith row by that factor and subtract I will get 0 there so that is why I first calculate the factor now I want to move from k equal to i plus 1 to less than n so k now is a variable which scans my subsequent rows below i of course I do not want to do this endlessly so how long will I do it as long as i plus 1 is less than n after all if I have reached my last row and I have made that coefficient 1 there is nothing below it so I do not have to do anything there I calculate this factor and because of this factor I know that the kth row ith column element is going to be 0 because after subtraction so I set it to 0 but remaining coefficients I calculate so I multiply kth row jth column and subtract from it the factor times ith row jth column that is the linear combination so what I am doing is I am calculating linear combination for j equal to i plus 1 to n minus so I have ith row I am now looking at the kth row in the kth row from i plus 1 i plus 2 i plus 3 up to the last element I have to do this linear combination this is what it does and at the end then I calculate the right hand side which is the kth element of b has to be again subtracted from the ith element of b multiplied by appropriate factor so you agree that this implements the Gauss technique of eliminating and if I go through all the values of i, j and k I would have in the end obtained my matrix in the desired upper triangular form all that I need to do now is to do back substitution so I start with x n minus 1 which is simply the right hand side n minus 1 th value and for i equal to n minus 2 to 0 I go backward now for every row I find the sum of the remaining elements using values of x which are already found out and using this back substitution I calculate sum in this iteration and I simply say x i is equal to b i minus sum and get me the remaining elements so this clear how exactly the matrix multiplication will work you may want to go back and examine the variation in the various indexes i, j, k and make sure that the process is followed correctly this algorithm is correct because I have tested it of course I produce the output just as I had one page of instructions for input I have another page of instructions for output so I will produce matrix a matrix b and I will put the result this facility we already discussed but I am just including it in the slides for the sake of completion whenever I execute a program I have to keep giving data again and again and again particularly when I am trying out to debug that program and if I have large amount of data that is required for every execution it is cumbersome to type that data every time I execute my program and therefore I use redirection by that I put the data in some file and use a symbol less than this is the formal way of defining that redirection so whenever we execute c in what happens is the data is collected from your keyboard every operating system has a feature by which you can tell the operating system that look when you run this program don't read the input from keyboard instead read it from a file similarly you can also tell the operating system that when you produce the output don't produce the output on my screen but produce it in a file so that I can later on also examine that file this is called input and output redirection in Unix it is done simply by using a less than symbol immediately after the program name so you say dot slash a dot out less than some file name immediately from this point onwards all your cn statements will be executed by reading values from this file so this file should better contain values in the proper format which are required as input for your program in exactly the similar way you can create an output instead of a terminal you can create it to some file if a file by the same name exists earlier it will be demolished and rewritten so if you want to generate outputs for different runs with different input better give different file name for your output file you can of course use both of them simultaneously so this is a perfectly valid command dot slash a dot out less than input data dot text better than output data file dot the names of files are your choice they can be any names which are permitted in cc++ so this I guess is known I would like you to start using it even in the labs I have seen that whenever you execute a program you type your values perhaps because they have few values it does not matter but imagine if you have to give 12 values for every execution and there are 5 iterations 5 runs of the program that you require before you debug it it is not worthwhile to type those values 12 times so 5 times so you might put them in a file and do that this is the sample input data file I will upload all these files in the Gauss program on to Moodle so that you can read that these are the input values the 4 by 4 matrix coefficient matrix this is the right hand side matrix minus 3, 3, 4, 6 this is the result so when I execute that I re-display the coefficient matrix and the values of the variables which are calculated as 1, 2, 3, 4 observe some idiosyncrasies of C out when you use C out to print values it does not print it in a manner which is easy reading for the human eyes for example it will print 0.5 here but the corresponding value here is 1 so it will print the next value 1.4 immediately it will not align it in any way it will rather keep printing values whatever space that value requires after that it will print the next value so this is not very pretty printing ideally the way we would like our output would be something like this doesn't this look better you have all the coefficients in an organized fashion unfortunately it is very difficult to control such output using C out you can we will study that particularly when we look at C out and the less less operator in the context of the formal object-oriented framework of C++ and study a whole lot of settings of the output field weights and so on we will see that the traditional way however in proper printing and similarly in proper interpretation of input data is to use two standard built-in functions into the C++ library which come from the C last standard library actually one is called f which is used to print properly in a formatted way in the output and the other is called scanf we are not going to discuss those functions now we will do that after mid-sem we will continue to use C in and C out statements so this is just an indication that there is something inadequate here that's it I have thank you