 We will start now. Today we have a rather heavy agenda. We have discussed the use of multi-dimensional arrays, particularly taking example of matrices. We have seen how simultaneous equations are solved using a matrix representation for the coefficients and the right hand side of the equation system being represented by a single dimensional array. Today we will see some more applications of multi-dimensional arrays. We will very quickly look at the matrix multiplication problem, which is rather simple and straightforward. Then we will look at another problem. This is about what we call magic squares, magic squares of certain properties. We will try to write a program to determine whether a given square matrix is a magic square or not. More importantly, we will then move over to some very real-life applications and usage of matrices in processing images. We are familiar with photographs, but when photographs are digitized, these can be stored as digital images and then there is a variety of way in which these images can be processed. Many of you may be familiar with software tools which are available. In fact, most of the modern cameras can directly do some of the processing that we will talk about. We will look at representation of digital images. We will look at the notion of histograms and we will solve one problem of enhancing the quality of an image by a process what is known as histogram equalization. It enhances the contrast of an image. We will see some examples. We will see how we can recalculate the values of picture elements called pixels and then we will write a program to implement this histogram equalization. Matrix multiplication is a straightforward operation. I suppose this is well known, right? If you have an M by N matrix A and N by P matrix B, then the multiplication of two matrices A and B which are called multiplication compatible matrices because there is a common dimension which is same. So you cannot have in general M1 by N1 and M2 by N2 matrices. They cannot be multiplied. But if such are the matrices, then the matrix product is given in terms of the following formula which evaluates every element of the resultant matrix. So if Cij is i th row j th column element of matrix C, the result matrix C, then this is given by sigma a i k into b k j and this summation is done over all possible values of k. Obviously, since k varies over the common dimension which is N, k will vary in our case from 0 to N minus 1 because there are N common elements. So you will get an M by P matrix here. But the processing that you will do since you have to calculate every element of M by P matrix, obviously you will have a nested iteration, one varying i from 0 to M minus 1 and inside it another iteration varying j from 0 to P minus 1. That way you will have M into P iterations being executed. Within every iteration now, you will have a third inner iteration that will vary k from 0 to N minus 1. Otherwise, it is straight forward. This is the segment of the program which does matrix multiplication. So we presume that we have read matrices a and b. As you saw from the formula, I will set up an outer iteration first varying i from 0 to M minus 1. Inside it, I have an inner iteration which will vary j from 0 to P minus 1. Now these two iterations will go over calculating every element c i j for all possible values of i and j. However, to compute every individual element, we need an additional iteration inside this. For each value of i and j, I will have to sum up that sigma over k of a i k multiplied by b k j and that is achieved by this inner iteration. So for k equal to 0 to N minus 1, I will calculate c i j as c i j plus a i k into b k j. This is a very straight forward summation. I suppose all of you can follow this. In fact, matrix multiplication is one of the most straight forward programs that you can write in c c plus plus. The important thing is to remember that having two nested iterations is not adequate. You have to run another iteration to calculate this summation. Matrix multiplication needless to add is required on many occasions in computational problems. This problem is not computationally relevant, but is very interesting. The problem of magic squares. So a magic square is defined as an n by n square matrix and it has the following properties. Sum of elements of every row and every column is same. Not only that, sum of elements of each of the main diagonals is also same as above. So look at this middle row. For example, 3 plus 5 plus 7 is 15. You look at this diagonal. 6 plus 5 plus 4 is also 15 and 8 plus 5 plus 2 is also 15. So when you get the same sum for all rows, all columns and both the diagonals, then you call it a magic square. This is of course a very ancient phenomenon has been known to practically entire mankind. Chinese had used it. It is called lo shu square there. There is a very interesting story about this. In fact, many interesting stories about magic squares in Wikipedia. So you can look those up. 3 by 3 magic squares were known in Vedic times to Indian mathematicians. In fact, they are more like occult kind of themes that the magic squares were used. Now magic squares can have absolutely any set of values. In general, the values are unique and in particular, if the values range from 1 to n square, then that is called a normal magic square such as this one is a normal magic square. It is a 3 by 3 magic square. 3 square is 9. All numbers from 1 to 9 are seen here. Of course, you can see that if you add a constant value to all the elements, most probably you will get another magic square but it will not be called a normal magic square. In general, it may not happen that you add a constant value to all the elements and you still get a magic square. You may get the rows and the columns sums right but one of the diagonals may not work out right depends on the configuration that you have of the numbers. The problem that we consider is to find out if a given matrix is a normal magic square or not. So, to design the algorithm and write the program, we first note some facts. Any n by n matrix will have n rows and n columns and it will have two diagonals. Now, if I want to test whether a given matrix is normal magic square or not, I will have to add all elements of rows, all elements of all columns and two diagonals but this sum should be same as all right. What should that sum be? Now, this is another property of normal magic square that we exploit. The sum of elements of any row, any column or any diagonal of a normal magic square is given by this formula n into n square plus 1 by 2. In fact, if you add up all natural numbers from 1 to n square that will be the total number of numbers there that sum is related to this formula you can easily verify that. Anyway, for a normal magic square we know in advance if the size is given it is n, then we know what should be the sum of every row, every column and every diagonal. Having determined this sum, we proceed to find out the sums of individual rows and columns. Since there are n rows and n columns, we will have to accumulate n sums for rows, n sums for columns and two sums for diagonal. These will be iterative calculated. In order to store these sums, we will need arrays of size n. Size n array is required to store row sums because n by n magic square will have n rows, n columns. I can of course define n variables but if n is unknown I cannot do that so obviously I use a one dimensional array there. Having calculated all these sums I will check if each of these is equal to the required sum or not. If it is, then it is a magic square. If it is not, it is not a magic square. This algorithm is mostly straight forward. There is one tricky part in the way in which the algorithm is implemented and I would like you to pay attention to that. This is I am presuming that we would not be testing for larger than 20 by 20 magic squares but of course you could always say square 200 by 200 if you wanted to. n is the actual size that somebody will give us. I j are of course my variables for iteration indexing and sum is a temporary variable to accumulate sum if I ever need it. However, I know that I have to accumulate sums for all rows and all columns so I declare an array called r sum, an array called c sum. These will hold values for row sums and column sums. Since I have only two diagonals, I am using two variables one called d1 sum and the other called d2 sum. So I will hold the sum of all diagonal elements of one diagonal in one and another in other. I first read the value of n and all the matrix elements and then initialize r sum and c sum. I have not written code for this. I suppose you are all familiar with how to do that. She read the value of n, have a nested iteration i equal to 0 to n minus 1, j equal to 0 to n minus 1 and read all elements of the matrix. Of course, either in the same iteration or outside it you can initialize r sum 1, r sum 0, r sum 1, r sum 2, r sum n minus 1 to 0, same way c sum and diagonals sums have been already initialized to 0 in the declaration itself. Observe this simple logic of calculating all the sums. I am running an outer iteration for i equal to 0 to n minus 1. Inside it, I am running an iteration for j equal to 0 to n minus 1. Whatever I do inside will be done in square times. Observe what I am doing. I am saying r sum i plus equal to square i comma j, c sum j plus equal to square j i. There is a trick here which we shall see. We claim that this summation when all the iterations for j are executed for i n j, not for i n j but for j here and later on for i, I would have got all the sums right. Diagonals require only n elements to be summed up and those n elements are on the diagonal. The first one is easy to note. This is outside the j iteration so when i varies from 0 to n minus 1, d 1 sum accumulates the total of square i comma i which is very sensible. But if I want to also calculate d 2 sum, given that I have a variable i varying from 0 to n minus 1, how do I do that? That is why I said this plus equal to what? Note that d 2 sum cannot be inside this iteration. It has to be outside because only n values are to be calculated and not n square values are to be summed. So let us look at individual details here. Consider the way rows are added. In any particular row, we have r sum i equal to r sum i plus square i j. This iteratively calculates that. For example, for i equal to 1, j equal to 0, 1 and 2, initially r sum i will be 0 and for i equal to 1, I have pointed out that I am summing up the central row here in a 3 by 3 matrix. So it will first take when j is equal to 0, it will take value 3. When j is equal to 1, it will take value 5 and when j is equal to 2, it will take value 7. At the end of this, it would have added all of these and would have given me r sum i. So this is very straightforward. Let us look at what happens when the column sum is calculated. Row sum is being calculated in a very straightforward fashion. We can understand that column sum however is not calculated in this fashion. Please note that this operation is happening inside the inner loop for j. So r sum i, the value of i is fixed by the outer iteration. That is why in the example we are saying suppose i is equal to 1 and i is equal to 1 is pointing to this row completely and the entire inner iteration of j will sum up r sum. So how are terms of a column added? Now the way the program is written, this sum is calculated partially. Every time you execute an inner iteration for j, no entire column is summed up. Instead imagine that for every element, there is an element of some array which is standing at the top. So for example when some value of i let us say 0, then for i equal to 0 this particular c sum j will accumulate in a square j, i. Since internally j is varying from 0, 1 and 2, first time when the inner iteration is executed it only adds 8, 1 and 6 to the corresponding c sums. Just the corresponding column elements of the first row. Then when you go out to the outer iteration, i is incremented and when i becomes 1, again in the internal iteration for j equal to 0 to n minus 1 it will add 3, 5 and 7 which are the column elements of the second row into the corresponding sums. And finally for i equal to 2 it will further add the remaining elements. Consequently the column sum is partially added to the array of c sum. So if this is c sum 0, c sum 1, c sum 2 for a 3 by 3 square, then in the first iteration for i equal to 0 the array elements will be 8, 1 and 6 because 0 is added to each of these. When i becomes 1 it scans the second row and corresponding elements 3, 5 and 7 are added to 8, 1 and 6 you get 11, 6 and 30. And finally in the third iteration for i you will get the values 15, 15, 15. So that is how the partial sum is being accumulated. You can of course choose not to do this summation in this fashion. You can sum rows separately by doing a nested iteration for i and j and you can sum columns separately by running a separate iteration. But this can be accommodated here without too much of complication and therefore this has been done. The more interesting problem is that of the diagonals. How are diagonal terms added? The diagonal addition is not specified in our program inside the iteration for j. It is specified instead just look at this. I have only r sum and c sum being calculated. The diagonal sums are both being evaluated outside the j iteration. That means whatever summation is specified here it will accumulate only n values from i equal to 0 to i equal to n minus 1. This is what happens for one diagonal. When I say d 1 sum plus equal to square i i is varying from 0 to n minus 1 to 2. I will get the first time square 0, 0 will get 8. The second time square 1, 1 will be added. That is 5. Third time square 2, 2 will be added. That is 2. So 8 plus 5 plus 2 is 15. So this sum will actually add this diagonal. The point is in the same iteration when i is varying from 0, 1, 2, 3, 4, n minus 1 I also want to add these elements. We can observe that these elements in terms of i are stipulated as let us say this is n minus 1, 0. In this particular case this is 2, 0 element. This is 1, 1 element and this is 0, 2 element. So you have to be careful in prescribing what should be the element of square that you would add to the diagonal 2 sum. Here is a small quiz. So while I am summing the main diagonal element some of the other diagonal d 2 sum can be incremented by the term what? I have an outer iteration. So just read these things. First choice is square n minus i plus 1 i. Second choice is square n minus i i. Third choice is square i n minus i minus 1. And the fourth choice is none of these. This sum must be done inside the loop for j. So anybody for a? b b can you raise your hands? a nobody 2, 3, 4, b 4, 5, c very large number d nobody. So everybody is sure that this sum will be calculated somehow. Let us analyze this for some specific value of n which is the easiest way to do such things. So let us take n is equal to 4. If we take n is equal to 4 i i will take value 0 0 1 1 2 2 and 3 3. Then the expression that was written here n minus i plus 1 n minus i and n minus i minus 1. Notice that I have inverted the choice that was originally given in the quiz. But that is symmetric. So it does not really make a difference. Now this n minus i plus 1 i will prescribe elements 5 0 4 1 3 2 2 3. Element 5 0 does not exist for n equal to 4. This n minus i i will prescribe 4 comma 0 3 comma 1 2 comma 2 1 comma 3. Again this is wrong for n is equal to 4 the largest element has an index 3 not 4. This is obviously the correct choice. This will go over elements 0 comma 3 1 comma 2 2 comma 1 and 3 comma 0. None of this of course is bunkum because you cannot have a diagonal sum inside the inner iteration which is executed in square times. The option that was given in the original c which I said is symmetric to this is also valid. I can either write this or I can write this which is what was written in the quiz d 2 sum is equal to d 2 sum plus square i n minus i minus 1. If I use this symmetric version then the way the elements will vary will be first 3 comma 0 then 2 comma 1 then 1 comma 2 and then 0 comma 3 would be added. But essentially it will still add the second diagonal proper. The entire purpose of this small exercise was of course one to introduce the notion of magic square which are interesting properties but more important how you should set up nested iterations, how you should keep in mind how many elements or values you have to calculate so that you do the proper summation within the appropriate iteration. So if you have to calculate only n values you should not be doing something inside n square iteration unless you are picking up elements alternate iterations or some such thing. In general you have to be careful about how you handle index expressions for your arrays and how do you handle these summations. We now look at digital images. Most of you are familiar with photography. You have all seen digital cameras, you have all seen JPEG pictures on the internet. Okay fine. Generally the pictures that we see in the digital form are a collection of what we call pixel values. A pixel is termed picture element. A picture, an analog picture as you see has printed on a photograph for example does not have these five lines. There are gray lines, expose the film and develop that film. However those are not discernible by human eye. What you do is in order to store these photographs digitally you sort of divide the entire photograph into certain number of rows and columns and at every intersection you say that there is a cell which is called a picture element. Now that picture element could have some color. That picture element will have some intensity of light reflected from that cell. These two qualities determine what a pixel is. Collection of all such pixel values represent that image. These are therefore arranged in some array say width by height m by n or w by h or whatever. So there are w rows and h columns or m rows and n columns. Now how do you represent pixel values digitally? There a lot depends upon what are the components of that picture element in terms of the light intensity and the chromacity of light. How many frequencies that light is reflecting? If it is a color photograph it is a different story. If it is a black and white photograph which has multiple gray values it is a different story. On the other hand you could have simple line drawing where something is either black or white but not a gray scale. The pixel values therefore are classified to belong to either one of these types. There is a one bit pixel. It is internally represented as m which is monochrome. Monochrome is not a proper scientific term. Monochrome actually means light of a single frequency. So there could be a red monochrome, a yellow monochrome, a green monochrome, a purple monochrome. Any particular frequency is monochrome if that is the only light that is reflected. However in that reflection you can have a immensely large number of intensities that are being reflected. Here in addition to a light being monochrome we are restricting the intensity levels to be either zero or one. So zero is black and one is white. However in the monochrome you could go to the next level where you say that I can discern between various intensities of light that get reflected from that monochrome picture. Such digitization is called gray scale digitization and typically the gray scale is divided between zero to 255 or you have 256 distinct values. The reason for choosing 256 distinct values should be obvious to you because these values can be represented in a single byte inside a machine. However it is not uncommon to have digitization which requires lower or more bits even in a monochrome light. So you have 3-bit pixels, 5-bit pixels, 9-bit pixels, etc. These are uncommon. However 8-bit pixels are the most common to represent gray scales. So here again zero represents black and 255 represents white. I have chosen to use black and white because invariably gray scale images are shades of black and white. But as I told you earlier that is not necessary. I could have a red monochrome for example which has various light intensities of red varying between zero to 255 and that could also be stored like this. The fun begins when a picture has all kinds of colors. Then you don't know how to represent them. Finally colors have been broken into three primary colors red, blue and green. And what you do is you take any pixel whatever is the color and light intensity you actually break that light intensity into three primary color intensities red, blue and green. You measure each one separately and store three values independently juxtaposed with each other. So you have one byte for red, one byte for blue and one byte for green. That means you require 24 bits to represent just one pixel in a color photograph. So this is the, this is termed as C type. So you have MG and C typical notation used in digital images. Now if you have 24 bit color you can actually have 16 million colors. Human eye on the other hand cannot discern so many colors. The typical capacity of a human eye to distinguish, differentiate between different colors is limited to anywhere between 200 to 2000 colors. So what does it mean? What it means is that actual picture may have many colors but what you see will be only these many. 16 million colors does not mean that you have actually a photograph which has all the 16 million colors because then you will not have any meaningful picture but you will just have pixels of different intensities, unique intensities. That is not how pictures are in real life. In any case what we want to discuss is given such digitized images. How can these be processed for getting a better quality image? So we look at our familiar arrays, two dimensional arrays again. I could store the pixel values of a digitized photograph in terms of row values and row by row I can store those values so I can have multiple rows. Obviously I will have w into h pixels stored into a two dimensional array having w rows and h columns or m rows and n columns. So I am now on familiar territory. I have a matrix m by n or whatever and every element of that array that two dimensional matrix will store one pixel or one picture element. This will not be true if I have a color picture then I will require a three dimensional array where the third dimension will store red, blue and green values appropriate. Monochrome or grayscale fingerprint images have small size. Fingerprint identification, fingerprint comparison is an extremely important branch of scientific computing and that is universally used for uniquely identifying an individual. You would be familiar with the fact that the national unique identity mission is actually trying to capture fingerprints for Indian citizens over the next few years. So how do you compare two fingerprints? How do you say that this fingerprint is of the same person that the other fingerprint is? These are really involved and interesting issues and last year in fact the entire CS11 batch worked on projects in fingerprint analysis, fingerprint storage, fingerprint duplication finding, etc. Note that if you have very large images you need to compress those images in some because the size will just, yes you have heard of cameras which are called 7 megapixel, 8 megapixel, 12 megapixel. All that they mean is a single photograph captured by that camera is capable of capturing 12 million pixels. Since it is usually a color photograph, a single photograph will produce 36 million bytes of data, 36 million bytes and if you have 1000 photographs you are doomed. Although the memory is cheap it is not that easy either. People therefore have defined variety of ways in which they compress this image. There are very interesting coding techniques which can be used to compress the image. The image can be compressed in a lossy fashion or in a lossless fashion. A lossless compression means that whenever I compress an image I guarantee that I can recover from that compression the exact original values from which I obtained this. So there is one-one-on-two mapping between the compressed image and this. However, so the compressed image is not necessarily in the conventional form of say P by P matrix if the original was M by M matrix. There could be variety of ways in which you compress them and in fact the format for storing compressed images differ depending upon the strategy of compression use. The more commonly used formats however are lossy compressions. That means some image quality is lost. However, that is where a judgment comes into picture. That if I find that while my camera might have got absolutely crisp photograph what I store in that lossy compressed format is slightly inferior to the original but it is perfectly acceptable to me then we tend to accept that format. The most common lossy format that has been accepted by practically everybody in the world is called JPEG. You are familiar with this term JPEG? Okay. But there are several other file formats. There is a raw file format. There is a PNG file format. There is a PMP file format or bitmap for short. There is a TIFF format. There is a JIF format. There is an XMP format and so on. So Wikipedia as usual has a huge letter of information about various formats including there are some cases where you can even find a C++ header. That means structure of the bitmaps or structure of the organization of the pixels is well defined in programming terms. We of course are going to look at the processing of images by the process choosing to analyze only those images which store grayscale values without any compression. So that means every picture element will be one byte long. I mean its value can be represented in one byte although we will use one integer location and that value will be between 0 to 255. So you conclude that picture element values or pixel values can be read in a matrix and the matrix elements can then be processed further. We already know how to find a solution to simultaneous equations. We know how to multiply matrices. Here of course we will have to do something different and we shall see an example of what kind of processing we do. We take it that each element of a matrix for a grayscale image would have a value between 0 to 255. Notice that type which you have defined which we had used earlier actually occupies one byte and since grayscale type is considered by C++ to be an integral data type meaning it is undistinguishable from an integer value. It is an unsigned integer between 0 to 255. It is therefore only appropriate that if we have such images we declare an array of care type variables two dimensional array. That will be very compact in storage since we are not very comfortable in interchangeably using care and int. For our examples we will continue to use int arrays. We now define a term called histogram. All of you are doubt very much. Statistical definition of histogram. Anyway some of you are familiar that is nice. In mundane terms it denotes the frequencies or count of number of times either some event or something occurs. So in case of images we talk of a histogram table that indicates how many times a particular value occurs in the image. By value we mean a pixel value. So effectively what we are saying is for each possible value the number of pixels in the image having that count will be a component of the histogram table. Since there are only 256 distinct values possible the picture may have millions of pixels. But every pixel value can be only between 0 to 255. The histogram table will have only 0 to 255 elements that is 256 elements. So a single dimensional array or a histogram table will accommodate the entire histogram for whatever be the large image. What does every element of that histogram table represent? The 0th element will represent how many pixels in the original picture have pixel value 0. The first element will represent similarly how many pixels have value 1. The 60th element will represent how many pixel values are 68 and so on. Why do we use this histogram? What is the advantage of having this histogram? We shall see that through an example. Here is a sample image. It is artificially constructed. We shall later on see a real image. It has 8 pixels. So you can identify some squarish things here. If you count them 1, 2, 3, 4, 5, 6, 7, 8 and there are 1, 2, 3, 4, 5, 6, 7, 8. A pixel in a real image will never look that big. This is artificial. A pixel is not even discernible by human eye. That is why you see a picture in its entire majesty. But for an example this will suffice. We are seeing 8 by 8 artificial example. 8 pixel by 8 pixel and each pixel has some value between 0 to 255. We notice some peculiar things. We know 0 is black. Black is in Z black. We know 255 is white. White as in sparkling white. Now notice this white for example. But I do not see anything which is similar to this white. The whitest part that I see is here. Obviously the intensity or the pixel value is not 255. It is more like 100, 150. We don't know what it is. But it is less. Similarly, I don't see a Z black anywhere. Perhaps this column, perhaps that pixel, this pixel are closest to black. But they are still far away from Z black. And all other pixels have picture values which are in between these two. What do we conclude? I have an image which has very little contrast. Contrast means extreme elements are present of the picture values in that picture. So this has less contrast. Let us examine what are the picture values or pixel values here in this image. So this is a sample image. The corner most pixel which we dubbed as the blackest sort of has actually a pixel value 52 which is not anywhere close to 0 as you can see. The highest value in this entire picture map is 154. So 154 is the whitest image whereas a true white should have been 255. We now understand why that picture looks slightly hazy. What would we like to do with this? This data is easy to collect. Either from a camera or from variety of other means you can digitize a photograph and you can get a data in terms of these pixel element values. Having got that, one of the things that we wish to do is to improve quality of the underlying image. Amongst very many techniques to improve the quality, one of the techniques to improve the contrast of the image is called histogram equalization. So let us look at the notion of histogram here. I have tried to superimpose that picture on top of that array. I actually wanted to make the picture transparent so that you could see all the values. Unfortunately the power point used does not permit me to do that. I have made a sort of peep hole here. This is the pixel which has value 154. If you could see inside this, this would have value 52. These are the histogram values. Remember how we defined the histogram? Histogram is a table or a single array of elements 0 to 255 and every element stores the number of pixels in the image having that value. Given the image that we have, we notice that there are no elements having values between 0 to 51. So therefore all those entries are not shown here. In actual table, all those entries will be there but they will contain zeros because there are no pixels. There is exactly one pixel having a value 1. There are three pixels having value 55. There are two pixels having value 58, three having 59 and so on and the largest 154 there is exactly one pixel. So the values are between 52 and 154 and that is why we have inadequate contrast. Well, we cannot enhance the contrast by arbitrarily moving these values left, right and centre. They have to be moved of course. What is the ideal contrast according to us? Ideal contrast is in an image which has jet black pixels and white pixels. So I should have a mapping from this image onto some new image which will have values between 0 to 255. But I cannot do that arbitrarily. I have to preserve the property of the original image. That means the relative values that you see here and the number of pixels of that particular value they must be related from the original to the final mapping. This is what is achieved through Histogram equalization. Very crudely speaking and we shall see that through the example what I want to ensure this is the Histogram. I have the original pixel values elsewhere. What I want to ensure is that the lowest value should not be 52 but it should be stretched to 0. The largest value should not be 154 it should be stretched to 255. Now when I stretch these values other values will also get stretched upward or downward. And what does that stretching means? That stretching means that the original pixel values must be modified appropriately to reflect this stretched value of Histogram. This is achieved by computation of another function called cumulative distribution function. This is what we wish to do. The Histogram is concentrated in a narrow range. The possible values are from 0 to 255. So we would like to stretch the Histogram and this is called Histogram equalization to get a better contrast. Ultimately what do we desire? Whatever technique we might name and ultimately we have a set of pixels. These are the actual pixel values. We want to map these values into some new value. Therefore we are looking for a function. Given an existing pixel value say some value V should be mapped by a suitable function to function of V. I call it H of V, H standing for a Histogram made function. If I derive an appropriate function and formula from the Histogram then I can apply that function for every possible pixel value that exists and calculate a new pixel value. I should get a new image and see how that image looks. To calculate this Histogram equalizing function we need to look at another concept called cumulative distribution function or CDF of a Histogram. If you are familiar with percentile scores in exams you would know what cumulative distribution function is. So if my percentile score is 90% that means 90% people are below me in the exam. If it is 92% that means 92% students are below me. In exactly the same fashion if the cumulative distribution function has a value 25 at 67 what it means is that for pixel value 67 or less there are totally 25 pixels. You can see that I can easily obtain the cumulative distribution function by summing up successively the elements of the pixels that I get. So these are the pixels. How many elements are there with value 52 or less just 1? How many elements are there with value 55 or less? 3 plus 1. 4. How many values are there with pixel value 58 or less? 3 plus 2 plus 1. So if I keep adding these I will get the cumulative distribution function. There is nothing fabulous about it. It is a very straightforward concept. Please note that all intermediate values also will have a cumulative distribution function value but that will be static at that point. So for example how many pixels are there with value 57 or less? There are still 4 because at 57 there is nothing. Estimulation function and I get all the non-zero values shown here. Of course you will understand that there are 0 to 255 values. This is the formula. This is actually given in Wikipedia but you can derive it almost easily and you can in fact derive a simpler version of this formula also by looking at what exactly is happening which we shall comment on in a moment. The equalization formula to calculate a new value for any pixel value V. Remember we mentioned H of V. So given some V how do I calculate the new pixel value? For that you actually calculate the CDF of V that is for that pixel value what is the CDF? The cumulative distribution function. Subtract from it the minimum CDF. In our case the minimum CDF was 1 divided by m by L minus CDF mean again and multiply this by the entire range of pixels minus 1. This appears complicated. Let the time being assume that this formula will yield an equalized image. We shall see the equation. We specifically look at our kind of image. For our image L or the range of values is between 0 to 255. 256 possible value is the range. So L minus 1 will be 255. We have an 8 by 8 image. m is 8, n is 8. So m into n is 64 and CDF mean is 1 because there was one element below 52 that we saw. We substitute these values and we get this formula. That means given any V, just find out what is the CDF at that V. Subtract 1 from it, divide it by 63, multiply it by 255 and round it off. Rounding means if a value is 5 or less you take the load value, it is 0.5 or more. Please note that you will get a floating point value out of this, a fractional value out of this. You have to convert it to the near by counting. So here is an example. CDF of 78 is 46. Cumulative distribution function for pixel value 78 is 46. That means there are exactly 46 pixels at or less than 78 pixel value. That is what CDF means. If CDF of 78 is 46, then a pixel value 78 will be equalized using that formula. This will give us a new map or equalized value. Now we know this formula. We know exactly how many possible values exist in the original picture. They are 0, 255. So why not I calculate an equalizer map sort of that if the original value was 0, it should be according to this formula this. If the original value was 1, it should be this. 2, it should be this. 3, it should be this. At this time I don't even know to bother what are the actual pixel value. I calculate this table completely, a one dimensional array and now I can use that to map my original pixel value into the new pixel value. So this is the example. I want to calculate where does a pixel with value 78 go. For that I will find out what is the cumulative distribution at 78, it was 46. I will calculate this formula. I get 0.714286 into 255. I round it up and I get 182. Observe that the pixel which was having value 78 is being stretched upwards. It is increasing in intensity to 182. Some pixels will move backward, some pixels will move upward but the equalization will be done in total harmony with the original image and the relative pixel value. If I apply that formula to each three pixels that I have, here I have 64 pixels but I could have 64000 pixels. I could have 12 million pixels. It doesn't matter. As long as every pixel is between 0 to 255, I can apply that formula. Notice what has happened here. This is the modified pixel map. What was 52 has become 0 now. What was the largest value? Some 154 or something. You will notice that it has become 255. All values have got sort of spread. Now if you look at an actual image composed of these pixel values, you will see this. Just look at the original and the new one. So this is the original. This is the new one. Actually the background is white and that is why these multiple whites are coming as almost stark white. That is not so. There is a contrast which you cannot see because of the lighting environment and the background environment. But you will agree that the contrast has improved. Let us look at another grayscale picture. This is some kind of a scene. Somebody has gone to Karnal or someplace and taken a photograph. So you can see a sort of hilly region. There are some trees here. And this region you can see does not have adequate contrast. And this of course has far more number of pixels than 64. This is probably a 1280 by 1000 kind of pixel map. Which means more than a million pixels. The formulation is same. The values are still between 0 to 255. You have the individual pixel values. You can calculate the histogram. Having calculated the histogram you can calculate the cumulative distributive function. The histogram and CDF are exactly 256 element arrays. And having calculated CDF you can calculate another array which is the mapping function. The equalizer array sort of. And once you have that array, every pixel from this can be mapped on to the newer verse. If you did that, this by the way is the histogram of this image. Observe that while the values of pixel should be between 0 to 255 all pixel values in that image are concentrated somewhere between 100 and 120 to 200. In fact this is the distribution. The cumulative distribution function is shown by this line. The cumulative distribution function is 0 for most of the time. Rises rapidly and then stabilizes. The reason why you don't have a contrast is that the cumulative distribution function is almost a vertical line. The ideal cumulative distributed function should be this. A straight line with unique slope. What does it mean? Effectively what we want to do is you take this straight line. You want to stretch the lower point to 0 and the higher point to 255. That is exactly what it means. If you did that with the cumulative distributed function the new pixel values and the histogram will look like this. So this is called equalization because what you are doing is you are equalizing the histogram across the entire range of possible values. If you do all the computations which are suggested by that formula you will get this image. Would you agree that this image looks better? I have just shown the process here. This is the original picture. This is its histogram. This is the cumulative distribution function. We calculate both. Then we want to effectively do this. But we will not be doing any computations here. We will rather directly be finding out the mapping array and then mapping all the individual pixels through that array here. And this is what you will get. There are a variety of tricks that you can do here. You can decide the cumulative distribution function of each portion of the histogram to be stretched. You can stretch only one portion this side. You can stretch only other portion this side. You can stretch this not entirely between 0 to 250 but between some. If you have played around with any photo editing software on any PCs you would see a variety of things that you can do there. In fact most modern cameras as I said have embedded software. You take a photograph and you can directly do this histogram equalization and before storing the final image in the compressed form because you are going to lose some information when you do that you can actually do histogram equalization. So the programs that we are now going to see is actually being written and is embedded into digital cameras and such things. Before looking at the program construction I would like to introduce an important concept. This concept is called associative arrays. We shall use associative arrays for histogram computation. Some of these names should be familiar to you. They are from your CS11 class. I have just put the roll number name. This is the lab batch number and this is the day of their lab. Suppose I want to count how many students are there in Monday batch? How many are Tuesday? How many are Wednesday? What will I do? I will go through this entire list and keep counting. Monday 1, 2, 3, 4, 5, then somewhere here 8, 9, 10. Then I will go over Tuesday, etc. And I will store these values in some array. So let us say this is that array. Let us say ultimately I get a count 91, 90, etc., etc. Unfortunately, when I talk of a machine array in C++ my index is 0, 1, 2, 3, 4, 5. However, I want to know these elements as Monday's element, Tuesday's element, Wednesday's element because my notion of tagging these values is to this day. What I want, therefore, is to associate one of the values which occurs in my data which I call a key value to become an index of an array. If I can have such a facility where the key that I am looking at or a value that I am looking at becomes an index of this array then I will call this arrangement an associative array. So this array associates a value or key as an index. Of course, we do not know how to look at arrays which have indexes as funny strings like Monday, Tuesday, Wednesday. There is nothing funny about it and in fact in this C++ course when we study strings we shall actually study an abstract class called associative arrays which permits precisely this to be done. However, for the time being we are familiar with arrays which have indexes 0, 1, 2, 3, 4, 5, 6. Can I have somehow these 0, 1, 2, 3, 4 themselves becoming keys in some data in which case I can use the notion of associative array because then I can use these for a table lookup. Any time somebody asks me how many strings in Thursday, I will say go to Thursday's element. If you recall our histogram problem, I do not have the funny issues of Monday, Tuesday, Wednesday but I have actually a 0th element, 1st element, 2nd element, 3rd element and I have exactly 256 possible values or key values in the actual data of million pixels. I can therefore do the following. In order to calculate the entries in the histogram table, I can use the notion of associative array. I will say that each index value here is associated with a key value that will be available in this data. This is my data. This is 8 by 8 for sample but as we just saw it could be million pixels, could be 12 million pixels. It does not matter because we know that inside these pixels there is no value which is less than 0, there is no value which is greater than 250. Consequently, every particular element here is a key to this particular array called what we may call this histogram. So histogram then, my histogram array becomes an associative array. Let us look at the computational issue. How will I compute histogram values? One very mundane way of saying, okay, let me say how many values are there at pixel 0? So I will set up a iteration for i equal to 0 to m minus 1 for j equal to 0 to n minus 1. I will go over this element, this element, this element, this element, etcetera, etcetera, etcetera and keep counting how many zeros occur. No, nothing occurs so I will put 0 there. If I continue doing this, look at the value 68. Suppose I have set up this iteration, I have done it once for 0, then for 1, then for 2. I have come to 68. What will happen in 68? I will look at this. Is this 68? No, 68, 68, no, no, no, no, no. I notice this is 68. So I count 1. I further go forward. I notice this 68. Count 2. Count 3. Count 4. And finally count 5. Array ends. I will put 5 here. You agree that this is a very straightforward way of computing histogram? But look at the computational effort. For every element value in the histogram table, I am going over the entire array. So there are 1 million pixels. I am looking at those 1 million pixels 256 times. But using the notion of associative array, I can significantly simplify this process. How? What I decide is, look, I will maintain these elements as if they are sums or counts. I will initialize these to 0. I will treat each one as if it's a variable called sum 0, sum 1, sum 2, etcetera, etcetera. And I will set it to 0. Now I will start looking at these elements. The moment I encounter a value, say 52, I will use this key as key. Go to the index 52 and increment that by 1. When I look at this element 55, I will go to 55th element, increment that by 1. That means in exactly one scan of this entire image array, I would have accumulated all the counts. Let us just quickly do that for another value, let's say 70. So I am looking at these arrays. Anytime an element comes, I would have accumulated that. Initially, let me say that I start 70 as 0 to begin with. And I am of course looking at elements. The moment an element 70 is encountered, I found 170, I will add 1 to this. I will continue going. Whenever I am discovering other elements, by the way I am similarly adding 1 elsewhere. But when I see the next 70, I will add 1 more to this. And then this. Anymore? No. So when I finish this, by this time this value would have become 4. And that is my histogram value. Do you see now the power of associative arrays? So if I can somehow use a increment value as the key and directly access a location corresponding to that key, then I can do increment, decrement, table lookup, whatever I want. This is what is implemented in our algorithm. Notice that I can extend the same concept. Once I calculate cumulative distribution function, and then I calculate the sort of reverse mapping from original pictures to this. Again I can treat the original pixel as a key, go to the corresponding mapping element directly, and take the converted value in that position. So I don't have to do much computations for the reverse mapping either. So this is a problem. I have not put a complete program here. I will put it up on the web. But essentially outlines what we propose to do here. It reads the pixel intensities, calculates the histogram, calculates the cumulative distribution function, and recalculates the pixel values such that the histogram is equalized. And this procedure is supposed to give us better contrast. So here is the program. I initialize all table elements to 0, histogram, CDF, and the so-called equalizer mapping function. Then I calculate histogram table entries. Observe how it is being calculated. i equal to 0 to m minus 1, j equal to 0 to n minus 1. Histogram of image ij element plus plus. That's it. Image ij is the actual pixel. If that pixel value is 70, 70th element will be incremented by 1. If that pixel value is 52, 52nd element will be incremented. For the power of the associative array we have been using the actual value as key as an index to this array. In a single iteration or rather single nested iteration, spanning all elements will get this histogram. So the value is out. You can calculate CDF table entries. CDF table is very simple. As I said, it is how many pixels are there with such and such value at or less than this. So I start with 0 being 0. Even if 0, there is some value it will get accumulated here. I know that I don't have anything at this juncture because at or below 0 in my present image there is nothing. However, I will do well. I am adding the previous ones. The 0th element of histogram is directly assigned to CDF. So if there are 4 pixels at 0, the CDF 0 will be 4. Then from 1st element onwards I calculate this and this summation will get me the entire CDF. Since for that formula application I require the minimum value, I will find the minimum value or not put the code here. Look at the entries in the equalizer. That is where that formula has to be applied. I have the CDF value. So all that I do is I find out the ith element of equalizer that is round off CDF i minus min divided by m into n minus min multiplied by 256 minus 1. Notice the use of the word float. This is called typecasting and I am forcing the type of CDF i minus min to become float. Otherwise what will happen? This will be integer, this is integer and this first division will collapse all your computation. Instead of getting a fraction it will get 0 or something. The multiplication by 256 minus 1 to 0 is still 0. So this is important that you convert this into float and then you round it off, you get the ith element of equalizer. In a single scan of 256 elements you will get all the equalizer value. Once you have the equalizer value, the reverse mapping is very steady. Again we use the powerful concept of associative. We know that the equalizer table is actually a mapping for a given picture value, pixel value say 36. The 36th element will tell you what should be the new pixel value. So again I go over the nested iteration over all the elements that I have in the picture value. I take every element, image ij is that element value. I use that as an index in the equalizer array getting the new element i. You understand this? Actually it is, but if we use the notion of associative array we can still get these results, but what we will do is we will treat this as some index k somewhere, some index j somewhere and then do that. But direct mapping is useful. Later on after the mid-sem we will see associative arrays in their full majesty when we can even have arbitrary character strings as index. Consider this. You have roll numbers. They are only 600 students. Suppose I have record of all students name, marks, this, that, that or some array, big array containing one structure per row. Now given a roll number I would directly like to express that particular row unless I convert that roll number into a number 0, 1, 2, 3, 4, I can't. Here I was lucky the values were incidentally same as the index values for my arrays. In proper associative arrays we shall see other things that can be done. It was the image of a fingerprint. Just to tell you that image when you take of a fingerprint you actually get a gray scale. And this gray scale image is the dirtiest thing to handle because the pixel values are all very funny. What is there in this fingerprint are called ridges and minutiae points. And these are the ones which are unique in every fingerprint. The general process which we may describe through some write-up which you can read if you are interested is to first thin this image. So you have these thick ridges, you know. There are so many blackish pixels here and whiteish pixels here. You convert them into one pixel thick lines. Then, so before that you may convert these into binary images, 0 and 1. Because after all you are looking at the fingerprint lines. You are not interested in gray scale. It is not some beautiful picture which you want to hand. You want to compare fingerprints with some other fingerprint. So these are some of the processing that you will do. It is just to show you an illustration. We will stop here. Okay, thank you very much.