 Today all this cannot be discussed right because it is overview of the session of machine learning. I will cover only 3, 4 popular model with the examples only right the basic definition and the examples. What are them? Under supervised we will understand one or two classification method and the regression models. Popular model of classification and the regression any unsupervised we will study what is clustering process and what is dimensional reductions or say couple of examples of under you know k-means clustering or say you know dimensional reduction process. All these 4 technique of supervised and unsupervised we will study only through examples and the common overview. Let us go one by one. Let us start with the supervised learning first. Under supervised learning I told about you will have a level the dependent variable the output variable. So, that level will be one of the part of your data sets, but that will define your model performance right like I told you right you have a data and say the y and then x1, x2, x3 kind of thing x4 different features are there right. We do not call as independent variable sometimes we call it as independent variable sometimes call it as features characteristics of your data, but in the different data thousands of data suppose you have hundreds of data. So, this is your level data this is your level data right level data. So, using this particular level data we will study the performance of your model. So, that will be used for your training and testing purpose all data will be like all this for say 70 percent will be used for training and the remaining 30 percent will be for testing this level column will help you in measuring the performance of your model how accurate the model is doing or performing. Here you can see the CAD right in super purpose learning algorithms level data sets are very important these data sets consists of data point that have been explicitly assigned a category or a level what kind of category CAD you want to identify the CAD image right that would be there are many image, but who to who is CAD and what are the features are there that will fall under the CAD category. So, you can understand based on the features based on the features you can see whether it is a CAD or not something kind of output or level decision variable can be taken. Similarly, SPAM mail SPAM mail whether the SPAM or the original mail right genuine mail. So, that is that level variable, but there are different features are there based on that features you take the decision. This level serves as the ground truth of the learning process that we have discussed earlier. During the training the model ingest the level data and analyze the relationship between the input features and the corresponding level which is the input features and the corresponding level remember that is the core part of super purpose learning. So, who is your level data and who are your you know input features that you have to identify based on this input feature and the level data you define your machine algorithm you ask the machine develop a good relationship among them and then the machine will learn train the data and learn the algorithm and come up with the best prediction that is it. Through the iterative process the model adjust the iterative internal parameters weights to improve the ability to predict the correct level of unseen data points. Unseen data points mean the testing level next level. This process is known as a model fitting you understood. Now, common features like common popular super purpose learning algorithms are the regression which you have discussed logistic regression I will discuss in the next class. The decision tree random for a super support vector machine there are many methods of super purpose learning are there one or two definitions I will cover, but rest of entire introduction to machine learning or deeper of machine learning I would recommend you to you know go through a good book that I have recommended in my list of predictive modeling predictive analytics and also you know you can learn in deeper of machine learning through different specific course of machine learning techniques. Now, since this course is a part of majorly on business forecasting in the predictive analytics we are just giving the basic overview of machine learning process what are the techniques or the classifications are there. Now, look at here the basic understanding is regression and here it is a classification both falls under super purpose learning, but here you define yes or no category, but here your regression you are fitting a line like you know dependent share coming, but here you are making a classification who will fall in this type who is fall on other type right yes or no die or survived kind of male female good customer bad customer these are the kind of different aspects and here is a you know different common model the raw data and the feature extraction from the data features you do the feature instruction they train the model and fit the model based on your level output right. And then new data feature extraction and then this this part you know predict and through the level whether say yes no you know why you are do not buy whatever 0 1 kind of thing predictions will come and level the model and come up with the best prediction based on the you know confusion matrix or different prediction models are there which gives you or a major accuracy are there which provides you the accuracy of your model. Now, as I mentioned for each of them I will give one example first is the regression here you see regression say one example of super purpose learning is the regression, we have discussed detail of that single linear regression and multiple linear regression. A single linear regression you have the basic definition of y equals to alpha plus say beta x right, where x is the input and y is the output and you fit the based on the data sets you fit your model and check which model is giving better accuracy based on square right or lower standard error. So, that is called basic regression and supervised model of regression, when it comes to the multiple regression you have more than one variables independent variable y equals to alpha plus beta 1 x 1 plus beta 2 x 2 say where two independent variable are coming say age and academic performance or say in work experience and the academic performance both will help you in getting a placement. So, the y is your level value or say dependent variable. So, this dependent variables will suppose here you can see here is a blood pressure, weight and height all this will define your say blood pressure. So, these are kind of two some extra or say age or may high weight heavy weight can change your effect impact your blood pressure. So, blood pressure is becoming suppose a level value dependent variable actual and predicted right and then you have a features weight and height. Suppose, suppose example as I talked about say you know academic record will help you to get a placement or performance. So, there are different examples you can find which will help all this fall will under say basic regression of simple linear regression or multiple linear regression. You can refer to the sessions of regression analysis we will get detail of it. Now, come to the classification type as I told about there are two way one is the regression another classification. This is also to some extent supervised learning, but it classify the data not exactly predict the value for a given x it does not predict the exact value of y rather it classify the data into binary kind of say 0 1 or classify the man or man like different type of you know differently whether give the loan or do not give the loan yes or no kind of thing classification are done that is also fall under supervised learning also, but here you define the level right here you can say the in supervised learning classification of fundamental technique and the process I mean given here you can define the class of the level which level it falls whether it is a good customer or the bad customer or whatever example you can think about right. What are the techniques decision tree logistic regression random forest support factor machine KNN algorithm name based method KNN and support factor machine are more deeper more practical more interesting, but basic definitions are you know basic methods are like logistic regression decision tree logistic regression is very important in business forecasting also. So, we will cover detail of logistic regression in the next session, but basic definition like decision tree random forest are also very popular machine learning techniques under supervised learning. Example spam filtering sentiment analysis of text data whether it is a you know review if you are if you are reviewing a data whether it is a positive feedback negative feedback on neutral sentiment that also you can classify the data in three category positive say 1 negative say minus 1 or say it is a neutral say 0. So, this way you can define your data right yes no whole kind of thing and you can come up with a sentiment analysis of twitter rate also when twitter rate or available freely before along must recover the you know twitter data we used to do many data analysis based on the available data and we used to we used to come up with many recommendation system through our sentiment analysis process of twitter data as a part of machine learning. So, you know so this helps you in getting getting understand of different type of classification method of supervised learning of machine learning. So, what is decision tree? Blabber let me summarize end to end enter aspects I have mentioned here, but let me summarize the entire process through this graph. In decision tree suppose you will have to go for a bidding process right first you go for a bid or you do not go for a bid there might be suppose you want to play football or not whether there is a rain or no rain if there is a rain you might play the football, but if there is a no rain you can think about the temperature of the ground right if there is too high temperature say 45 degree 50 degree you may not play the football, but with a low temperature we do not windy whether is there you can play the football also. So, yes no different type of root we are creating of your decision making right that is called decision tree first whether you want to play football or not and whether temper based on the temperature or the weather. So, you are classifying the weather. So, this is called node sub tree sub tree and node. So, there are also bidding process example that I was talking about you go for a bidding for a new project from your company right and you go for your IPL player cricket player. So, you go for bid bid or no bid no bid means initially you are taking the decision, but initially the final decision is that you go for a bid or not that you decide from the root end of the root from the leave node you will find take the final decision, but it will be reverse channel, but how you are developing your new root that will come from the beginning this people define like this way also this way also people define this way also tree can be defined in this way tree can be defined in the you know vertical manner also in different book people use different flow of tree, but the first point are called the decision node which is called the root node root node root node and then intermediate sub tree will come because you are taking a decision based on the yes no yes no temperature of the ground and the high temperature low temperature medium temperature any weather. So, they are different situations we are classifying that tree and like you know if you go for a bidding whether there is a computer or not whether if there is a computer what is the chance of that particular computer will come up with a bidding amount and then different competition will come. So, you are dividing different root and finally, you are taking a decision right whether you should hire the cricket player or not or you know go for the project or not and you can do the market research also. So, that will come inside in between whether if you do a market research whether there is a while or not or you know whether they gold on the or say uranium are there or inside the or lithium is there in the ground or not in Kashmir. So, that different example I am talking about right or Bombay have there is a while or not petroleum products there or not that decision how will do you will do the survey. So, that is an inter decision process right you can put everything in your decision tree. So, start what starting point will be go for bidding or not, but over a period of time you do a marketing or you go a seismic survey and then based on that is the survey is positive you go for digging or bidding or if you do otherwise you do not go for bidding and then again there will be how many computers there are what are the computers of bidding price could be you do not know and accordingly you have to come up with a game planning. So, all these things can be put on a decision tree and in a reverse manner you can take a decision and you can come up with a good decision making process this is a supervised learning. Here you can see the general structure you can see decision note will come as a root note the first point final decision intermediate steps will be the right from different branch sub tree we call it as sub note decision note or the sub tree and last note in each tree you can see the last note we call it as a nip note nip note. So, there is a final decision yes yes no suppose here it is no suppose for example, yes it could be yes it could be no. So, how many ESS are there how many based on that you can take a decision majority you can take a final decision or based on the final outcome what could be the probability chance you can assign also and you can take the final decision over here whether it is a profitable or not the bidding or the decision making. So, this is all the decision tree look at here one example I have given suppose hiring decision or your job performance whether you want to take a job or not like suppose you got offer. So, there are three root three top terminology root note decision note and the nip note nip note will come in the end and root note will be in the top right or the beginning the starting point where the initial decision is made based on the salary offer below $50,000 leads to reaction if the offer is less than so annual offer say less than say $50,000 set for the sake of the example right consider this. Then you reject the offer otherwise you accept the offer. So, this is the initial point. So, suppose your offer salary offer is less than say $50,000 you reject the offer. So, decision tree ends there that is your nip note final note you stop there and you take a decision in the root note. Decision has to be taken in the root note first initial note you have to go back and final decision you are taking there, but how you are creating your tree. So, you stop there, but if the offer is more than $50,000 you have to accept the offer right, but there also you can divide suppose yes, yes offer is more than $50,000 you are not taking a decision you are creating more root more decision making more branches of your decision making what are them that how much distance it is from my hometown or say you know from my location it is a suppose you stay in heart of Mumbai city or say Bangalore city or say Delhi city or say Chennai city. Then you take a decision is it far away from my location if there is a nearby offer from I say city center or say you know I T half why I should not take the offer from there rather than going 4 hours early every day. So, you take a decision say. So, there also you are taking a decision in case you have more offer say and then also you know far away from your city you have to go to outside country that also you can extend this example to any other example also if it is nearby your country or say you do not want to go to say US say suppose you want to settle in India. So, then you take a decision whether the company is giving offer to India or in abroad you take a decision similarly and then suppose you know if yes then decline the offer if it is a say distance is too high or it is not giving the offer inside your country you do not take it and then you take that decision say you know or you can think about the if it is a foreign offer then you can go it is up to you right and then again this way you can classify even if the offer has a free like you know free offer like you know say incentives and leaves like additional leaves are there or not like you know early kind of thing or say you know additional holiday leaves or say kind of incentives are giving to you or not tax benefits are giving there or not or job opportunity or there or not additional feature career opportunity or there or not that also you can put only offer each one the money is not important your career or the interest in the job the type of work that culture that also very important the brand of the company you can classify and one by one you can take the offer final decision will come automatically you will get a decision and that will be a final route of your decision making so this way decision tree are been defined there are many examples which you can use look at it is a benefits of the structure approach it provides a transparency it provides the flexibility in taking decision making because all options you are actually examining and you are the best option will come as a final route of your decision making that is it but remember three component who is root note root note is the initial point from where your decision making starts and who is leave note the final decision making point the last point of your tree there the leaves are there there you are taking the final there you are getting the offer and that you can take to a final decision making through your decision note or the branch and then you take the final decision next the random forest imagine there are many application of random forest or many way of defining I will define the most effective or the most easiest way of defining the machine learning technique supervised machine learning technique of random forest what is that effectively in the previous slide we have discussed the decision tree one tree the one decision making one final offer or final decision making right but you have only one tree that you have a branch of the tree but it is ultimately one tree right you have taken a decision based on the sample data or the decision making process but random forest is nothing but you are entering from a tree to a forest you are entering from a tree to into a forest there are many tree are there many tree are there many tree are there and based on the majority of the forest majority of the community your decision will come not only from the one tree you take the decision you enter into the domain enter into the community take a different opinion of the people and then you take the decision that is called the forest random forest why random because you shuffle the data or main data the main data you have right main data you shuffle into different bag different sample random sample and from each you develop a tree and you take a decision and take a decision then you might say there are so many decisions finally because each tree will provide suppose you develop 10 tree then you have a 10 decision making right in each of the tree you can take a one decision you can use some decision making process inside that tree also you are you are open there is flexibility to take a decision inside your tree also whatever the method of machine learning or different techniques programming technique you can use right but ultimately you take a decision from one tree then from another tree based on the different sample because you change the sample size like you know size may be same or maybe or even change the size sample or size also but you are taking a different data right data may be overlapping of sums are there but you are taking different data not the same sample and you are taking another decision like suppose who will win the upcoming general election ND or NDJs say or who will be the PMSA so therefore you can take a decision you can take a sample based on the large sample you take a couple of sample random sample and then from the random sample you develop one tree based on the opinion and the strategy and one decision will come then again another tree another decision will come whether you take the offer or not job offer different type of illustration you can do now when you have a 10 tree 10 decision making comes based on the random sample data so what is the final decision then here you take the majority among the 10 tree if say 7 tree says yes and 4 tree say no then your final decision of your random forest is yes because there is a majority of yes if say 7 people say as a 6 people say no this product is not good and say 4 people say this product is not good or say bad or e good so you take a decision whereas the majority so this is called the random forest and the majority through decision making process it is just extension of decision tree in decision tree here is the example you can see look at here the definition the primary objective random forest algorithm is to classify the data into distinct group using multiple decision tree classify the data look at classify the data the main objective the main priority of you know primary objective of random forest algorithm is nothing but you learn the one decision tree technique then for method or the process you extend it to multiple sample the data and use different decision tree different tree in your forest and based on that you take a decision based on the majority classify the data into different class like different sample as I said sample and then based on the distinct groups using a multiple decision tree you take the final decision here you can see the example look at here your main sample you have classified into sub samples different random sample and you take one tree decision tree here final decision what is the outcome of this tree these are the branch intermediate decision tree branch I told right and these are the leave node and these are final decision what is the final decision here say the customer is loyal look at the example customer consider a e-commerce company interested in predicting customer charm whether customer leave the store or the service or if the customer will remain there with you. So, this you want to study based on the customer past purchase behavior of the customer. So, that you study whatever the model machine learning model you can use or the techniques that you can use you do supervised unsupervised you do whatever right classification clustering regression whatever you want to do you do it the logistic regression then association rule ABC classification whatever you want to do do based on that you take a decision whether customer will leave the store it will there the charm rate you want to calculate for a customer right or for in general. So, then you can see collect the data from the various features of his customer such as demographic usage pattern customer service interaction. So, all these things will be a feature right look at that and then based on that you level the data the customer will charm or not this is the decision making final decision because the level data right the charm customer will charm or not. So, now data sets you can divide the data into part then subset the data and classify into different subset sample and the train the data through decision tree and take the decision interval prediction from the each decision tree finally, what based on that from each decision tree suppose you have a three decision tree here say three decision one is the loyal loyal front, but where is the majority loyal has the majority. So, final decision of your random forest is the customer will be loyal that customer will not leave the store that is what random forest this gives this is a super set this is a super decision making more powerful decision making than the decision tree because decision tree is based on the one sample decision tree and the outcome of decision tree comes there might be a less of less prediction in terms of final accuracy because it is all about prediction based on the training and testing data you are coming up with the algorithm prediction model you set up the model. Then for the new customer when you apply it it has to be a accurate model right the accuracy level should be very high then only the company will adopt your model right because you have the data sandwich you based on the data you created the model. So, therefore, in this process of machine learning of supervised learning only decision tree people use, but random forest is much more powerful because here you take the majority opinion not only from one tree you take a decision you enter into the forest and you take the decision of many trees and then the majority comes up as the final decision making that is the benefit of random forest here you can see the advantages the unseen data point you can take and then you know majority vote comes and the final decision is that whoever wherever the majority belongs that is your final decision making you do not take any decision based on the majority come up with the prediction right this is what the random forest. So, far we have discussed the supervised learning concept and two-three example of it right the methods of machine learning or algorithms we have discussed. Now, let us discuss little bit about the unsupervised learning techniques also. So, the first what is unsupervised learning? I have already discussed basic difference between supervised and unsupervised, but we will understand through Lehmann examples here. The basic difference between the supervised and unsupervised learning is that it will have unlabeled data sets. In supervised learning what you do you define y upon x you have a supervised learning whether it is of a value or it is a level data right level data. So, this is what your supervised learning but in unsupervised learning you do not have any level data only the feature data and based on these data you classify the data you segregate the data you put cluster the data you do the factor analysis that data belongs to which category is it say children or adult it is a man or woman it is a this community or that community it is a employee or outside customer. So, this type of classifications are done or to some extent in a clustering are done we should not use the word classification rather we should say it is a clustering are done based on the feature data unlabeled data node level variable the output variable or the supervised decision variables the which are been derived or explained by the independent variable concept of regression and the classification will not be there whether it is a value or level value that is fine but that is used only for supervised learning in unsupervised that y part the dependent variable part the level variable part will not be there explain relationship will not be there only the feature data independent only feature data you classify them you cluster them segregate them and come up with the different data behavior pattern which will help you in taking decision specifically in the industry of understanding consumer behavior data pattern different category that you can understand and accordingly you can take a decision that falls under unsupervised learning you might say that then how the machine comes over here here also you have different technique different algorithm through which you can classify the data one or two I will discuss but you can understand the importance of unsupervised learning where you do not have the dependent variable or level data but only unlabeled only features are there based on that itself you can take a decisions you can come up with a good interpretation about the data pattern look at here you have a mixed data here right and then you do the classification or you can say the segregate the data and different type of you can you do not know how we will do that right machine will understand the data pattern through centroid method or K-means clustering. So, dimensional reduction system will come up with different you know kind of clustering or factor analysis based on that they will segregate the data into different category look at the category. So, nice of category can be done through machine learning of unsupervised learning algorithms and which will help you or the company to take a decision how many people fall under this category how many people are buying more product how many people are buying less product but how with high volume how many product people are regular all these things classification can be done right this is the advantage of unsupervised learning. The goal of unsupervised learning algorithm is to uncover the inherent pattern inherent pattern and the structure within this data without explicit guidance system will learn and system will come up with the best you know segregation of the data clustering of the data clustering is one of the most important technique which is been used as a part of unsupervised learning to facilitate the data and the exploration of the pattern and the structure within the data sets because here if only the feature data you do not have any level data right dimension reduction technique is one of the method which is very popular where you might have a many features right many features you have but you cannot use all these features and analyze them right you reduce the dimension reduce the number of features through intermediate process electronic transformation all these things and then you take a decision and come up with a better prediction because you can handle that important features rather than taking all these features and then you can come up with a better prediction that is called dimension reduction. There are many example like you know of unsupervised learning like market segmentation, anomaly detection, recommendation system all this falls under to some extent unsupervised learning application. Now, let us understand the basic two basic concept one is the clustering say k means clustering that examples only and say say you know dimension reduction right principle components. This two basic popular method we will discuss there are many methods like agglomeratic clustering db scan fcm gmm there are many methods are there we will understand the basic k means clustering which is very popular and the dimension reduction. In dimension reduction there are also many methods but we will focus only on say you know principle component analysis. Now, in clustering you can understand classify the data into different classes right and different cluster you get look at here how the data can be clustered into different partitions say cluster one cluster two cluster three this way but all these techniques are done through centroid process. Look at here suppose here you have a data and different sample data and based on that what will be your k value number of clustering right what will be the number of clustering the case election that you can do through your initially you randomly define say one cluster two cluster and then based on that you know do do your centroid classification like distance from the data sets distance from the data sets distance from the data sets distance from the data sets and check who is giving most suitable distances like minimum distances Euclidean this distance you can use and then you come up with your base selection. The first step is that you know look at the elbow where how many classification you want to do one classification there is no clustering then right only one group you are taking about then you take two group and you see like you know how many clusters like data are being featured and what is the distance among all of them that is the best cluster or not or overlapping are coming or not then you could take one more like you know clustering say 3.123 say now you have created a three cluster say and based on three cluster you have seen the distances from the different other points to these three clusters are less so after that you can do four cluster k equals to four also you can decide through the algorithm through the central distance process and then from the different all the data now instead of 2 now you are considering 4 right 1 2 3 4 and then you decide the centroid of all of them centroid of all of them which will be your main point and the distance from them and after that the centroid will also be moving centroid right it is not fixed at a later stage it will be fixed but you do not know who will be the final center point of each cluster. So, now this process you do iterative process of distance from all and then segregate the data let us move and finalize you know who could be becoming the best cluster. So, once look at the elbow process we call it here after certain point of class like you know clustering finalization like number of cluster finalization you will see there will be no distance no major differences are coming the distance almost fixed or no classification further group you are not able to create. So, once you you can put your threshold value once you realize that there is no further classification or the clustering are been not possible almost segregation or the different clusters are been done the patterns are been identified you stop there. So, k is selected k is selected there once k is selected you put the data into different category and take your decisions and you know develop a decision and centroid process you can use and the distances among them you can find and who could be your major representative of each cluster that also you can do using the centroid process. The detail analysis you can refer through different book and you can get to know about the k-means clustering, but here you can see one example like k-means clustering is an unsupervised learning techniques that identify the group the clusters of similar data points within the unlabeled data set that is the first part there will be no level data all are unlabeled all are features based on that you are doing different type of features classification right like say children adult senior the groups are you are developing right children adult senior. So, this different groups we are creating through clustering process, but initially you have a data. So, what you do you just take any point define your k value and see the who could be their best cluster whether k equals to 1, k equals to 2, k equals to 3 and take the distances over them through centroid distances always you try to find the distance is point that you have found and you check who is the best cluster what is the best cluster once that is done k-means clustering are done and that will give you and then the data falls under this group because you have classified the data centroid distance are being finalized and based on that you do the features analysis and you see what type of customer what type of people what whatever the example you want to do recommendation system you can do suppose this data follow orders are senior customers. So, you can now make a recommendation to this the email id the contact details are there with you and you can write a mail to them share or madam you buy this product suitable for you for the children's you can provide them chocolate facility or kind of you know different type of toy you can provide or the recommendation you can provide right for the adult you can put the different people shirts pants all these things you can you know different makeup products you can provide to them also. So, the advantages you get from the classification or say clustering process under unsupervalued there is no level no expand relationship you are defining right but here based on the features data unlabeled data you are doing a cluster different there are many metallic factor analysis also you can do also here but we are not focusing that we are just focusing the basic k-means clustering example you can see a retail company want to analyze their customer data purchase history demography etcetera to understand their behavior and the customer pattern base k-means clustering can be used to do that. You can define three customers say you know budget conscious home improvement and the high spender based on that k-means clustering you know in the algorithm process the centroid distance minimization process and finalizing the base centroid this three cluster you can find budget conscious say say for example say home improvement and say high spender you can think about these are the high spender these are the budget conscious and there is a home improvement whatever the example you can fit and you can come up with the outcome and the recommendation system enhance customer satisfaction target segmentation the sales marketing also you can do based on this understanding the data behavior pattern this is the advantage of k-means clustering this is not a supervised it is a unsupervised learning because there is no level no supervised causal relationship are being developed here now the last part last model that we want to add as a part of basic information of unsupervised learning that is called dimension reduction why dimension reduction is popular because in general what happens because in these days too much of transactions or too much of data are available in our day to day live right every day huge amounts of data are being gathered whether it is a social media whether it is a facebook whether it is a instagram whether it is a youtube google or you know daily transaction of your different company more data are being stored rare in that case what happens there will be many features forget about level data we are not discussing only the features data there will be many features many column of your data many column of your data maybe hundreds of columns will be there can you take all of them and analyze the data and understand the data behavior etc sometimes it will become very difficult so in that case what happens people use dimension reductions so among the data who are the most important you are not selecting them only you are not filtering them and rest of you are not discussing that is called feature engineering that you are not discussing but here also interrelationship among the multicollinearity we are not discussing them here we are seeing that among the data suppose you have a 100 data points the column among them we will find the system will find some you know conversion of the data and the new level will come new features will come through latent analysis variable classification through to some extent covenience matrix and eigenvalue the majority will be decided and then you can reduce the dimension from 100 from 100 dimension to say 10 dimension this new dimension you are developing but this 10 dimension you can define as a latent variable intermediate variables who will have a new data sets none of this data direct data will come here none of this direct data may come may not come generally does not come here will be new data sets new component like you know you take a data sets say data and you take a ln logarithm right so if you take ln say logarithm of the data what happens you get a new data right data transform these data sets transform into ln ln data right this is changing the scale same logic here so this transformation will not be the same data sets you have a new data component so these transform transform data with 10 component will represent the entire data sets you do not take this data into your final decision make or the analysis process you take this data this because it is easy it has been reduced with dimension and this does not reduce the generality of the original data that is the advantage of dimensional detection enter part is a clear mathematical analysis first to you know define the covariance matrix and the first you normalize the data and then normalize the data all the features data normalize each of them by subtracting you know mean and x minus mean say mean by sigma and for each data you do that and then do the covariance matrix covariance matrix of the all the data sets and then you define eigenvalue calculate the eigenvalue the eigenvector and then do the majority of them and then you finalize your number of how many like you know cumulative data sets you will define that how many values or new dimensions will be the in your final decision making process. Suppose 100 will come also 20 will come back to 800 will come back to 15 or say 10 good no you have only 10 now out of 100 or say 150 and now these 10s the dimension could be different the data point could be different but through this process first you normalize each and then you calculate the covariance data and then you know calculate an equation like you know calculate the lambda and the corresponding eigenvalues and you do the measurement like cumulative values of them how many eigenvalues will have a higher weight as a first 10 will take the 95 percent of your data the data features that it stop there are 99 percent stop there remaining 15 20 or say out of 30 data they will take only 2 3 percent do not take that the new data new eigenvector that you found based on that you take your final dimension say first 7 8 9 10 will take care of the 90 percent of the data behavior stop there and take that based on your cutoff point stop take that new dimension data to your final analysis and take the decisions your generality look at that preserve the essential information your you are not changing the generality the original behavior of the data that remain there but in the intermediate process you are reducing the dimension by converting into a new scale that is what the dimension reduction there are many methods like principal component analysis PCA and LDL linear discriminant analysis there are many T SNE there are many methods are there but you know PCA is very popular let me explain that I have already explained but you know we have a data sets with the following features is individual very basic example I am sharing with the dimension reduction for PCA principal component analysis what is that suppose height one feature look at the features how many features you have data let me see height and then weight and then say gender and then say income and then say education level right how many features you have 5 features suppose it is a too big suppose and you want to reduce the dimension. You use your PCA the process as I told first you convert the data into normalization process right and then first you do the normalization of the data say data and then call consider the covariance matrix big covariance matrix among the all the data sets after normalization normalize you are changing the scale right normalize then the covariance data sets and then you calculate the eigenvalue like a like you know and then next next step you calculate your eigenvalue of the data through chemical emulsion equation then the eigenvectors you calculate for each eigenvalue for each eigenvalue you calculate the highest eigenvector and then you calculate your cumulative value say you know by summation of these data sets like percentage of presence the weight based on that you define your number of dimension that will be the final dimension reduction value or the final dimension news data news scale and that will go to your final decision making and the recommendation system and overall process here you can see you have 5 features here right. Suppose we have done the dimension reduction and we have come up with the 2 dimension here out of 5 now we will take 2 none of them are coming as individual but we have categorized into 2 components what is the first components suppose interpretation first components might be represent the combination of height and weight height and weight will give you one new dimension and what is the gender income and educational level will give a another dimension. So, we have 2 dimension dimension one is here and another say here what are them new latent variable you are getting right what are them intermediate variable you are getting what are them new dimension what is one is the say it is say the body size let me delete this we can see one dimension you have kept new dimensions a overall body size right intermediate variable one of your dimension reduction and another you are defining say the second principle component might be say culture combination of say income education and representative yeah gender may help you in defining your economic status socioeconomic status. So, now you have developed you have reduced the dimension from the 5 features to 2 features one is the overall body size now you are not considering weight and height you are considering the body size as one feature as one dimension one intermediate variable which will help you in taking a decision about your final data sets and come with the recommendation or whatever the decision you want to take and the another dimension another variable intermediate variable that you have found what is that the socioeconomic status remember gender education and the income are not using now you are not using, but the 2 variables that you have developed through dimension reduction the overall body size may be some some new parameter new variable new number will come but you are not losing the generative of height weight but the body size will represent the representative of original data height and weight. Now in this body size the new variable that you have taken take that as one principle component the another principle component could be socioeconomic status which is a combination of these three through this you know mathematical process of dimension reduction you follow and come up with your another dimension principle component principle component that is called socioeconomic status. Now you have 2 principle component one is the body size and the socioeconomic status remember it is easy to predict discuss in the in the platform or the discussion panel or wherever and you can take a decision also this helps just example I am giving this helps the process of dimension reduction here I have given only one basic layman examples but when you go to the deeper of machine learning and dimension reductions entire process you understand the entire steps of dimension reduction the calculation of dimension reduction in python you would not understand because just you put and then you will get the select the you know feature and you get to know the outcome of dimension reduction but here I have tried to give the basic explanation with little technical steps and the layman understanding of dimension reduction this helps you in taking like when you will large amount of features characteristics data sets it is very difficult to handle everything all of them and to take a decision you reduce the dimension and go for final decision making or prediction or recommendation or whatever analysis you want to do further you can do that this is a part of unsupervised learning part of your to some extent dimension reduction and clustering process this are the two method that I thought of discuss as a part of your unsupervised learning what is that K-means clustering and principle component analysis or say dimension reduction under and so what we have discussed today let us summarize now we have discussed now the basic definition of machine learning applications and the overall aspects and then the the terminologies of machine learning and then two category supervised and unsupervised under supervised we have discussed regressions detail next class we will discuss the logistic regressions we have discussed the application of them we have discussed decision tree and under classification and random forest now and under unsupervised learning there are many methods too popular we have discussed with example basic example one is the clustering K-means clustering another is the principle component analysis as a part of dimension reduction there are many more methods but overall this is what the overview of machine learning algorithms or machine learning techniques or introduction to machine learning but if you go deeper of all of them with the techniques applications coding and etcetera that will give you the further boost of machine learning introduction to machine learning or the confidence in machine learning techniques but we are not discussing that because our objective is to give the basic information of the basic overview of machine learning algorithms as a part of predictive analytics and as a overview of under the umbrella of business forecasting more detail you can learn through different sessions or different courses of machine learning techniques with that let us conclude today's session of overview of our introduction of machine learning in the next class we elaborate the logistic regression in detail with example thank you