 Hello, this is professor C. A. Murthy from ISI Kolkata. And professor Sukhendu Das from IIT, Chennai. We are both responsible for the lectures of pattern recognition in the NPTEL phase 2 program. Thank you. Thank you. Good morning to you all. This is the lecture on, we are going to give 40 lectures on pattern recognition. The responsibility of this lectures is given to professor Sukhendu Das, IIT, Madras and myself. I am C. A. Murthy from Indian statistical institute Kolkata. Before I start going into the details of the subject pattern recognition, let me actually tell you the uses of pattern recognition and why it is necessary to learn pattern recognition. For that, let me tell you a few examples. I start with the example of finding criminals or identifying criminals from photographs. Let us say you are an inspector in a police station and you have records of the criminals. Let us say the number of such records is of the order of say 1 lakh and at some place a new crime was committed and someone has taken a photograph of the criminal who had committed the crime. The photograph comes to you. Now your responsibility is to see whether this photograph is one among the existing photographs that you have in your database. Now let us say your database has 1 lakh photographs. What is the meaning of having 1 lakh photographs for each individual or for each criminal there? You would say you would write the name of the criminal, maybe the height, the sex, the fingerprints, the photograph and also the types of crimes that he or she commits. So for each criminal probably there is a page giving all these details. So if you have 1 lakh criminals you have 1 lakh such pages. So now you have one photograph so you would ask probably one of your people to get all these 1 lakh pages which are put in a file, the file size may be like this. So you would take the first page, look at the photograph and the photograph that is given to you, no it is not matching. Take the second one it is not matching, third it is not matching. How many times you are going to do it? Note that your eyes after doing this sort of checking for around 50, 60 photographs your eyes will lose the sensitivity to differentiate between two photographs. Then in such a case naturally a human being is sort of incapable of going through all these 1 lakh photographs to find out whether the criminal is existing in the database or not. So you would like a machine to do this job. Now how does a machine do this thing? This has simply too many components. The first thing is this information has to be recorded, it must be stored as a file that is the first one and like that you need to store the photographs or the information about this 1 lakh criminals. Then after that what you are going to do is when a new photograph comes you are supposed to do the matching. Now how does one do the matching? For each photograph are you going to go through all these photographs, all these photographs in your database, is there any way you can reduce or you can say that these photographs I need not see them, I need not check them, is there any way that you can do this thing? Yes, you can do this, for example say the height of the criminal that is the photograph that is given to you, say the one who has taken the photograph if he says that the height of the criminal is say less than 5 feet 6 inches, he is not really tall, his height is less than 5 feet 6 inches. Now in your database whose ever height is more than let us just say 5 feet 7 inches you need not look at their photographs, right. Secondly suppose the person the one who is there in the photograph is a male, you need not go through the photographs of all the females. So basically what you are doing is that you are putting some constraints or you are finding the features of features corresponding to this photograph so that in computer language basically you are making a tree, in computer language basically you are making a tree corresponding to this database so that you can do the identification faster, you can do the identification faster. Now note that I used one word feature, sex of the person is a feature, height is a feature, you can have features of many types for example the color of the hair, the color of my hair is gray that can be a feature, color of eyes, color of skin apart from height, weight and there are many other such things, you may also have colors, you may also have very many features depending on the problem at hand. The problem that I told you is known as face recognition problem, in the literature on pattern recognition this problem is known as face recognition problem. Let me tell you some more examples, one example is regarding classification of pixels in satellite images, I hope you are aware of what a satellite image is, the satellite takes the photographs of earth depending on the type of satellite and depending on the uses, the resolutions of the images that it is taking it varies, India has a satellite Indian remote sensing satellite sorry, it has several versions IRS1A, IRS1B, IRS1C. Let me talk about one version where the resolution is 36.25 meters that is each pixel on an average occupies 36.25 multiplied by 36.25 square meters area, on an average each pixel occupies this much area, usually these images are multi spectral images that is for each location on earth you have four images taken at the same time, these images correspond to the wavelengths blue, green, red and infrared, blue, green, red and infrared. Now your problem is for the whatever pixels that are there I hope all of you understand the meaning of the word pixel, I was mentioning to you, I was asking you a question whether you know what a pixel is, I assume that you know what a pixel is, so I am going to proceed further, the main problem in satellite, one of the main problems in satellite images is classification of pixels, you are given the photograph of a region on earth. The photograph may consist of let us just say it has 512 rows and 512 columns, so basically it has 512 by 512 these many pixels, 512 is number of rows, 512 is number of columns, so usually they have these many pixels and as I was mentioning to you every location it has 4 images corresponding to every location that is you have, these are known as multi spectral images and using those 4 images you are supposed to classify each pixel to one of the land cover types. What is the meaning of a land cover type, a land cover type is may be the pixel is corresponding to a water region on earth, the pixel may be corresponding to a hilly region on earth, maybe there is a building at the location of the pixel, so these are the land cover types. Now how many land cover types you may have, well there are simply too many, you may have water then among the pixels corresponding to land you may be having these pixels corresponding to vegetation where you have rice fields and then you can have open spaces that is barren land, you may have one particular category which is known as concrete structures where you have buildings present and you may have forests, you may have hilly regions, you may have snow like if you go to Himalayas or some such places you may have snow, you might be having sand, if you go to the desert area you will find sand, similarly if you go to the sea area there also you will find sand. So here I mentioned some land cover types, then the question is for the given region under consideration how many land cover types it may possess and once you know the land cover types, the number of land cover types, then the next question is how do you classify each pixel to one of the land cover types, before I come to the procedure of classification there is a valid question why do I need to do it, what is the use of doing this classification, well it has many uses. Let me tell you some of the civil uses, the government may be interested in knowing how much of forest area may be getting depleted every year because that has long term impact. Now if you ask human beings to do it, due to several reasons you may not be getting accurate estimate of this, someone may want to give, may want to say that more forest area is getting depleted, someone may say that though quite a bit of forest area is getting depleted he may say that well the area that is getting depleted is not really much depending on one's own bias towards the specific situation. You might be interested in planning something, for example here in Chennai you are planning to construct or may be the construction has already started about a metro, surely when people did this thing, when people have made this plan surely they must have looked at the soil content where at what place you should build this thing whether you have barren land or not or how much money is to be given if you have to find that particular thing okay and when you want to make a route then you should see how much time it may be taken so basically you need to know the complete geology of that location. Now if you want to do it on a massive scale then again you need to do, you need to use machines to do it and a massive scale many of these things you need to have machines to do it that is very much necessary. Now this is one use there are many uses from the point of view of a military, suppose you want to know in the border areas the amount of construction that is taking place in your neighbouring country this every government would be interested in knowing since it has implications on the defence capabilities of a government. So you would really like to know what concrete structures are what new structures are coming up in the border areas now it is difficult for you to send always people to find out what new construction are taking place rather if you take photographs of those places and find it out automatically it will be helpful to you and like that one can think of many many uses how much of how much area is devastated in the floods how does one know it you might have the photographs of that region before the floods you might be having the photographs of the region after the floods and you can look at the differences between them you can do pattern recognition to find out the amount of area that is that is that was devastated by floods it may be because it may be floods can be one of the things you might have earthquakes you might have drought like that you can have several other such natural calamities for which you will be needing this sort of information satellite images is one example where you need to use pattern recognition let me tell you there are very many other examples one another example that I would like to mention here is example corresponding to the human intelligence look at this this is a chalk piece okay now you see what I am doing I get down the stairs steps I just put the chalk piece on the floor the chalk piece is now there on the floor now suppose I ask anyone of you they are myself suppose I have to walk on this floor now what I would just do I would just walk like this okay I do not I know that this is a chalk piece and I also know that it is not dangerous so I just walked over this now I know that this chalk is not dangerous but instead of this chalk suppose there is a snake here then surely I would not walk like this surely I would not walk like so basically I have made a differentiation between what a chalk is and what a snake is the purposes of a chalk and what a snake can do and look I would make a decision about whether I should cross or I should move over the chalk piece or I should move over the snake now suppose instead of me a human being you want the same thing to be done by a robot robo t robot now how does a robot do this but before we go about how does a robot does this the main question is why do we want to make a machine to do this job there are several reasons for it let me tell you one of the reasons suppose you would like to send a rocket or rocket to Mars Jupiter Saturn or any one of these planets now surely you cannot send a human being to that place because the atmosphere there whatever it is it might not be suitable for a human being so that a human being may die so initially you would like to send a machine to those places to investigate to understand about the general nature of the soil general nature of whatever may be the atmosphere etc. Now you cannot send a human being there so you would like to send a machine there now when a machine is placed in that alien region on a planet this machine is supposed to navigate the entire region it should be able to move from one place to another so how does this machine move that is the problem for which I was giving you this example now let us say how does a machine go you see even I am asked to move from one place to another on this floor I just moved there are some steps I just went up the steps and I just came down now when a machine when it sees the whole scenario first it needs to know what is what that means it needs to know that this is a camera this is a TV screen this is a bulb here and this is a table this is a laptop that is blackboard all these things it needs to know it needs to know the uses it also needs to know whether any one of these things is harmful to it it needs to know somehow all these things now how does it do it how does it know it now a whole a host of literature is there on this problem you have the whole of image processing computer vision and so a few other allied subjects depending on the applications at hand you have many subjects which are trying to tackle this same problem the main subjects are image processing and computer vision and you have a few other subjects your medical imaging is one subject where people where the doctors and the image processing people they are trying to see what is happening inside the body there are many machines available which take photographs are which sense the which sense what is there inside our body you have x-rays you have several types of scans you have ECG EEG etc okay and all these things earlier in fact in India even now also the human beings would interpret all these things and they would give their judgments about the properties are about the diseases of the human beings under consideration but then these are very costly if you have a machine which does once the photograph or once the scan is given to the machine if it automatically says that at this place this portion is not proper so that is to be rectified if a machine says this thing then the we will be spending less on what a we will be spending less on this rather if you want a human being to say this thing then on the doctor will be spending more money whereas on machines would be spending less money this is about medical imaging and we also have as I said the usual recognition of objects like what is a table what is a chair how does one define what a table is how does one define what a chair is how does how do we know that what is a chair what is a table apart from this thing apart from all these objects one also needs to know the relationship between these objects for example let us just see for example in different rooms the location of switches that is at different places you will be having generally when you enter the room in some rooms the switchboard may be put on the right hand side in some other room the switchboard may be put on the left hand side of the door okay now whether it is put on the left hand side or the left hand side in whatever side it is it is basically a switchboard that is what this robo should understand if you are looking at what all the things this robo needs to understand you will actually look at the whole gamut of what we see and understand you see the whole the entire earth whatever objects that we see you want a machine to know what that object is this is only one part of it there is another part look at this table this is hard this surface is hard I know it but look at the carpet or look at this place this carpet is not hard so when I walk on a carpet I will be slightly more careful than when I walk on a hard surface when this robot is put in an alien conditions in same Mars on Mars or say Jupiter or Saturn one of these planets if it needs to move from one place to another place it needs to know that the place that it is moving the floor it is not soft how does a robot know it how does a robot know it suppose you are asked to stand in front of a hill and then you are asked okay would you like to climb the hill then you would say well I think I can climb the hill because I am sure that I can come down the hill whereas some person may say no no no I do not want to climb because I may not be able to come back properly I may not be able to climb the whole hill also properly these decisions we make without actually doing the job now here the robo in on an on a planet say Jupiter Saturn or Mars whatever may be the planet it must be able to make the decision when it looks at the terrain it needs to know whether there is a valley there or a hill it needs to understand that it is a hill or a valley and once it understands it it also needs to make a judgment about whether it can move up the hill or down the valley so that if it is moving down the valley it must be confident it will be able to come up now all these are actually part of pattern recognition where the main input to the system is a set of images the main input to the system is set of images if you look at the problem of identifying each and everything that we see you will understand it is the entire gamut of what pattern recognition is and if I say that you need to know the interrelationship between the objects that also need to be stored in the computer then you would see not only just the identification of the objects but also something more look at the name pattern recognition you are supposed to recognize patterns and the patterns may be existing at many many places if you look at the gamut of the set of problems that I mentioned you would see practically very many subjects coming into the whole spectrum of the subject pattern recognition what is the meaning of very many subjects coming into it let me ask you a question how do we solve a problem in mathematics how do we think pattern recognition people want to model the thinking process pattern recognition people want to model the thinking process the human brain there is a space there is a subject which is known as neural networks where the first algorithm of neural networks which is actually known as single layer perceptron or in those days when it was introduced this algorithm was known as perceptron it tries to mimic the thinking of human being so that it does the classification. So if you want to also understand how a human thinks and that you want to put it in the brain of a computer then what is it that we want to achieve in this subject pattern recognition we actually want to achieve everything that a human being can think logically we want to achieve everything that a human being can think logically can make logically what is logic the word logic has several implications let me tell you the general implications that we have in computer science literature basically the logic that we follow there is what is known as binary logic 01 a statement is either true or it is not true there is nothing in between that but let us look at human beings is there a statement is it always true or is it always not true does not there exist anything in between that for a human being it is not necessarily true that something is true or false only these two possibilities that is not the case for a human being there is generally something in between these two possibilities for in many many applications now let me give you an example let me give you an example you say that he is a good man but is he really good always you would say no or is he bad always and you would say no then what is this good and bad you are not giving complete completely bad that is 0 to him are completely good you are not giving the value 1 to you are giving something in between 0 and 1 like that in all human in all dealings of human being we tend to use these sort of adjectives and we actually have our own logic also when we use these adjectives for example when I am teaching I am teaching in one specific way my statements need not be always mathematically precise but the people who are sitting in front of me they do understand even if the statements are not mathematically precise but for a machine you have to always give precise and perfect statements so there is a difference between what a machine needs and what how a human being things so even this gap we would like to bridge in pattern recognition then you would you try to look at what are the what is the complete gamut of problems in pattern recognition I think I gave you the users of pattern recognition now the next topic I will try to go into how does this subject work I will go into the mathematics part of it slowly yeah I was mentioning to you the whole gamut of problems in pattern recognition so let me tell you the gamut of problems human beings have eyes to see something and to know what is what human beings have ears with which we hear and we understand what the other person is speaking and we also understand who is speaking depending on the pattern of speech we have knows which smells and by smell we understand quite a bit about the object under consideration and we also have tongue which tells you the taste of an object okay and we also understand whether an object is hard or soft when we touch it we also understand whether an object is hard or soft when we touch it so let me just write hard slash soft these are the five basic senses of a human being with eyes we see and understand what an object is with ears we hear and understand what the person is speaking and we smell an object and we understand some of its properties we taste and we also understand whether an object is hard or soft by just touch these are the basic five senses of a human being now once we get input from these five senses the input from these five senses it goes to our brain and our brain does analysis and it gives some output we want to put this whole process in a machine apart from all these things human beings have one another characteristic what is that human beings learn human beings learn what is the meaning of learning well there are several examples for the meaning of learning okay let me just give you just one small example this is my fourth or fifth time I have come to Madras this city Chennai suppose a person does not know anything about Chennai and he is given an address and he is asked to find he is asked to go to that particular location find that particular house or flat and this person has never been to Chennai then how does this person find it if you ask this question to any one of the people then even though the person has not been to that specific place under consideration the person is confident he will be able to find it out how the person is confident that he will be able to learn about that particular thing even though he has not seen it he has not read about it this thing this confidence and the process with which this person goes about learning about that particular phenomenon or that particular thing under consideration this is an example of learning and there are in fact very many such examples of learning so this sort of thing also we would like to put into the computer this is also we are understanding about the patterns that is existing in the nature so once you put the process of learning and once you put all these things then basically what is it that we are trying to do we are actually making a machine which somehow is replacing human being human beings apart from all these characteristics they also have one another characteristic that is we people are emotional so when we are emotional many times we may not be able to think logically now if you make a machine and if you make it without any emotions then that machine would be a much better machine than a human being it does not have the negative characteristics of a human being this is the ultimate aim of this subject the ultimate aim is to make a machine basically it will be a computer which has all these I should say faculties because these faculties human beings possess and you would like to devise one such machine so that it can walk it can speak it can hear it can see and it is able to know how an object smells and it is also it must be also it must also know whether an object is hard or soft all these things you would like to number one simulate number two model in the machine this is the ultimate goal of this subject pattern that means practically everything under the sun it is a part of this subject practically everything under the sun is a part of this subject and look at the aims of the subject enormous you would like to recognize patterns wherever they are in whichever phenomenon so the aims are enormous there are naturally several several subjects which came out of this which have become independent subjects now like image processing computer vision neural networks fuzzy logic and other subjects like medical imaging some things regarding biomedical engineering okay and if you look at the thinking process modeling a thinking process that is extremely complicated and till now the number of good results on this is really very very small the subject is still in a nascent stage so it is enormous and you have too many too many users of this now looking at the whole gamut of problems what is it that we are going to talk in these lectures looking at the whole gamut of problems the literature that is existing in the subject is in mathematics knowledge in mathematics terminology it is only epsilon I hope you understand the meaning of epsilon epsilon in mathematics is used as a very very small quantity it is greater than 0 but it is very very small quantity that is looking at the whole gamut of problems the results that are existing they are really very very small it is only epsilon now within those epsilon in our lecture series we will cover epsilon I hope it is clear let me tell you once again within that epsilon in our lecture series we will cover epsilon that is which is really very very small and very insignificant portion the number of journals associated with pattern recognition and other these allied subjects it is at least 1515 if not more at least 15 if not more international journals so you have vast amount of literature available on several different problems in this subject so with this brief introduction I hope I have made you understand what are the what the users of this subject are in my next lecture I will start going into the mathematics portion of the subject thank you