 Friends, today I will introduce a new topic in the multivariate analysis, this is called principal components or principal component analysis. In principal components, so basically in any study in which we are dealing with the several variables, one of the prime concern is to look at the variability of the variables. That means, in those random variables how much variation is there that means, how much variability these variables are contributing to the model. For example, you may have a regression model, you may have a time series model where variability is of importance. Now, in case the random variables are the variables under considerations or many then we may have to discard some of the variables because it may not be feasible to study all the variables at a time. In that case, it may be convenient to consider certain linear combinations of the variables which are more influential in terms of variability compared to other. So, the problem of principal components is the determination of linear combinations of random or statistical variables basically which have special properties in terms of variability. So, for example, the first principal component is the normalized linear combination which will have maximum variance. So, we can order them like we can put the one which is having the maximum variance. So, that linear combination let us consider say for example, a 11 x 1 plus and so on a 1 k x k. So, this could be having the that means, the variance of this is maximum then you consider second maximum a 11 x 1 plus and so on a 1 k x k second maximum and so on. So, basically what happens that in many of the practical studies, it is found that some linear combinations will contribute more that means, they will have almost 90 percent or 99 percent of the variability and very small variability will be there in all their components. So, basically what we can do that we can consider this as a new coordinate system we can call it z 1, this one we can call it z 2 and so on. So, in place of the original variables x 1, x 2, x n, x k we can consider new variables z 1, z 2, z k. Now, out of this it may happen that only z 1, z 2, z 3 they are contributing say more than 99 percent of the variability. Then we may discard other variables and we can consider our relationship of the variables among these itself. So, so basically this is the you can say in a nutshell the problem of principal component analysis. Now, let us develop the mathematical procedure to derive this. The procedures for finding out principal components originally we are discussed by Harold Hotelling, a American statistician in around the year 1933 he discussed or he derived this methodology. So, we are considering let x be a p dimensional random vector with covariance matrix say sigma. Now, this may have certain mean vector say mu, but without loss of generality I can shift it to mean vector 0, because in the discussion of the variance covariance matrix the mean does not play any role, because if I shift all the observations by the same amount then the variability does not get affected. For example, if I look at the simple property like variance of x plus a that is the same as variance of x. So, in general if I consider the dispersion matrix of say x vector or the dispersion matrix of x plus say some c then this is same. Therefore, we can consider without loss of generality take expectation of x to be 0. So, this will simplify in terms of calculation if we take this to be 0 otherwise we can go with mu also, but when the calculations I am presenting they will be easy if I use this one. So, let us consider let beta be a p by 1 vector such that beta prime beta is equal to 1. What does it mean? It means that the norm of beta norm of beta is 1. So, actually I will consider that linear combination which will be normalized here. So, now you take expectation of beta prime x. So, naturally that is going to be 0 because I have assumed expectation of x to be 0 here. So, this is going to be 0. Let us look at the variance of beta prime x. So, by the formula for the variability it is equal to beta prime sigma beta. This is the formula for the dispersion matrix. So, if the dispersion matrix of x is sigma then if I consider any linear combination of this then the variance of that will be beta prime sigma beta. So, what is our problem now? I want a linear combination. So, this is a linear combination because what is beta prime x you can consider it as say beta 1 x 1 plus beta 2 x 2 plus beta p x p. So, our aim is to find out those values of beta 1 beta 2 beta p such that the variance of beta prime x is maximum, but of course we have put a condition here. We want the normalization here that is norm of beta is equal to 1. Why that normalization is required? Because we can make it free from the units of the measurement here. So, we can consider here we want to find beta such that variance of beta prime x is maximum subject to the condition norm of beta is equal to 1. So, we can consider the method of Lagrange's multiplier. So, we consider a function say phi function that is equal to beta prime sigma beta plus lambda times 1 minus beta prime beta. This is the simple Lagrange's multiplier term here. So, this is the phi function and if we consider here this is del phi over del beta that is equal to twice sigma beta minus 2 lambda beta. So, we are putting this to be equal to 0 here. Let us call this function 2 and this is 3. So, you can take out this beta. So, this is becoming sigma minus lambda i beta is equal to 0. Now, you look at this equation you are very well familiar with that is A x is equal to lambda x basically this is becoming the Eigen value problem here is not it. So, this is Eigen value problem. So, to get non-trivial solutions we must have determinant of sigma minus lambda i equal to 0. Now, this is nothing but characteristic polynomial is not it characteristic polynomial in P this is of degree P and this will have. So, this will have P roots. Now, P roots can be denoted by lambda 1 lambda 2 lambda P. So, we can actually choose lambda 1 greater than or equal to lambda 2 greater than or equal to lambda P. See there is nothing wrong in naming them in such a way that the largest one is called lambda 1 then the second largest is called lambda 2 and then the lowest one is called ok there is one problem here. How did I put this here? Because you know that in generally if I find out the characteristic values are the Eigen values of a matrix then they can be real or complex is not it. Then is it possible that I can do this? It is not necessary is not it then how did I write this? That means, there is some assumption here. Since sigma is a covariance matrix I can make the assumption that it is positive semi definite. Actually in general we can have positive definite also, but all the time positive semi definite is always assured. If it is positive semi definite the Eigen values will be non-negative. So, this is possible since all lambda i's are positive or basically non-negative reals ok non-negative real numbers. Now, we do some manipulation here let us take 4 here. In the 4 1 you can consider pre multiplying by beta prime in 4. So, we will get beta prime sigma minus lambda i beta is equal to 0. This will become now scalar here. So, this is giving you beta prime sigma beta is equal to lambda beta prime beta. Now here there is an advantage here I chose beta in such a way that beta prime beta must be equal to 1. So, this is simply equal to lambda this is simply becoming equal to lambda. Now that means, I am having an expression for the solution here that means, what should be the value of lambda it should satisfy beta prime sigma beta here. So, basically what I am saying is that and what is beta prime sigma beta it is actually what was our original aim to maximize this quantity. So, I considered the maximization problem as an optimization problem using the Lagrange multiplier here I am getting the solution. So, what I am saying here is that this value lambda is actually the Eigen value here ok. So, what I am getting here here I am getting beta prime sigma beta is equal to lambda that means, I have solved the problem. What is the solution? The solution is that I choose the Eigen values and take the largest one is not it. So, the problem is solved here. So, this shows that this shows that beta satisfies 4 and beta prime beta is equal to 1 then variance of beta prime x is equal to lambda. So, for maximum variance we take the largest characteristic root that is lambda 1 corresponding to this we can find out the Eigen vector or the characteristic vector here. So, let beta 1 be the normalized solution of sigma minus lambda 1 i beta is equal to 0. Then the corresponding beta 1 prime x this is the normalized linear combination with maximum variance. So, actually if you look at this can be called as the first principal component. In fact, if we can say that if sigma minus lambda 1 i is of rank p minus 1 then there is only one solution only one solution to sigma minus lambda 1 i beta is equal to 0 and beta prime beta is equal to 1 ok. So, this is the first principal component ok. So, this this one is called the first principal component this is called the first principal component. Now, what is the next problem next we need to find a normalized combination beta prime x that has maximum variance of all linear combinations, but uncorrelated with the first one ok. So, this is the second this will be the second principal component. So, that means we want covariance between beta prime x and u 1 equal to 0. Now, this is equivalent to expectation of beta prime x into u 1 minus expectation of beta prime x into expectation of u 1 is equal to 0. Now, expectation of beta prime x is anyway 0 because of expectation x being 0. So, this one will not matter. So, this means basically we are saying expectation of beta prime x u 1 is equal to 0. Now, this is equivalent to beta prime now what is expectation of this one u 1 is beta 1 prime ok. So, you write it here. So, this is becoming beta 1 prime x ok. So, these two terms I am multiplying this is a scalar this is a scalar ok, but what we can do if it is a scalar I can write it in the reverse way also I can write it as beta prime x x prime beta 1 ok. So, this is just a simple mathematics here, but what this will give this will give you the sigma. So, this is becoming beta prime sigma beta 1. So, what I am getting the condition that this is equal to 0 ok. So, this should be equal to 0, but what is sigma beta 1 if you remember your original derivation here sigma beta 1 was lambda 1 beta 1 is not it because that was the solution of the first one. So, this is equal to beta prime lambda 1 beta 1 let me call it 7. See this lambda 1 you can keep on this side. So, what is you are getting? So, you just look at this condition here see this is equal to 0 lambda 1. So, that is non 0 anyway beta prime beta 1 is equal to 0 what does it mean? That means, the condition of uncorrelatedness how did I start with the condition I started with that a next linear combination which should be uncorrelated with u 1 that condition is reducing to that this new combination would be orthogonal vector to this ok. So, it is leading to that condition now. So, the condition of uncorrelatedness is reducing to finding a an orthogonal vector ok. So, that means, what is the problem now? The problem is beta prime sigma beta is to be maximized subject to the condition that you have the normalization thing. So, this term is the same as I wrote earlier this term you can see it is the same, but now I will write one more term that is minus 2. So, let I am just putting some minus 2 mu 1 this is just for adjustment term here ok this is coming from here ok. So, this should be equal to 0. So, I am putting that here. So, this is these are two Lagrangian multipliers now in place of one because two conditions are coming here. So, this nu 1 this lambda these are actually Lagrangian multipliers let me call it function phi 2 here. So, you are having del phi 2 by del beta that is equal to twice sigma beta minus twice lambda beta minus twice nu 1 sigma beta 1 that is equal to 0. Now, you can consider pre multiplication here pre multiplying by say beta 1 prime in the above equation we will get twice beta 1 prime sigma beta minus twice lambda beta 1 prime beta minus 2 nu 1 beta 1 prime sigma beta 1. Now, here we can use some condition sigma beta 1 is nothing, but lambda beta. So, this one and this one is getting adjusted here. So, you will get here it is simply equal to minus. So, minus twice nu 1 lambda 1 using 7 and that is this condition that I wrote here. This condition and this condition if we write here that is sigma beta 1 is equal to lambda 1 beta 1. So, let me call it equation number 10 here. So, nu 1 is 0 and beta must satisfy the condition number 4 that is again less sigma minus lambda i beta is equal to 0 because here this I am putting to be 0. Now, I am putting this to be equal to 0 then this is becoming same here. This 2 will not come because I have taken out this 2 from here. So, this will not come here 2 I am writing outside. So, beta must satisfy 4 that means, lambda must satisfy this condition number 5 here that is this because it is again becoming the Eigen value here must satisfy 5. So, lambda 2 will be the maximum of lambda 1 lambda 2 lambda p such that there is a vector beta satisfying sigma minus lambda 2 i beta is equal to 0 beta prime beta is equal to 1 and 7. So, these 3 conditions will be satisfied. So, we call this vector as now we name this vector as beta 2 ok. The first one I call beta 1. So, now this is the second linear combination that is u 2 is equal to beta 2 x this is the second principal component ok. So, what is the property of the first principal component? I discovered the largest Eigen value of the variance covariance matrix corresponding to that what is the Eigen vector? That Eigen vector gives you the combination that is beta 1 prime x then then. So, that will have the largest variance because the variance is that largest Eigen value itself then you find out the second one and then second one will have the Eigen vector. So, basically what we are doing is we are orthonormalizing those Eigen vectors that is you can actually use a Gram-Schmidt's process also. So, the second one can be continued. So, this procedure is continued at the r plus first step we want to find a vector beta. So, that beta prime x has maximum variance of all normalized linear combinations which are uncorrelated with u 1 u 2 u r. That means, we are saying 0 that is equal to expectation of beta prime x u i as we have already written that how this term is coming this term is coming from here that covariance between beta prime x and u 1 that was coming as an equal to 0 this condition gave you expectation of beta prime x u 1 is equal to 0 because of this term being a any way 0. So, if we use the similar thing here this one is actually becoming equal to 0 which is nothing, but given you beta prime x x prime beta i that is equal to beta prime sigma beta i and that is equal to lambda i beta prime beta i for i is equal to 1 to r. So, our aim is now to maximize we want to maximize now term let us call it phi r plus 1 beta prime sigma beta and the Lagrange's multipliers will give you now there will be r plus 1 Lagrange's multipliers for i is equal to 1 to r you will be getting this condition here where lambda and nu 1 nu 2 nu r are Lagrange's multipliers. Now, you consider the vector of partial derivatives. So, you will consider del phi r plus 1 by del beta that is equal to twice sigma beta minus lambda beta minus twice sigma nu i sigma beta i. So, this 2 will also go outside we are putting it equal to 0 say let me call this function as 12 and this function as 13 here. So, pre multiply by beta j prime if you do that then you will get beta j prime sigma beta minus lambda beta j prime beta minus beta this is summation here sigma in the previous term I had this after differentiation that with respect to beta if I do then this term will give me the this one. So, that is coming out to be twice sigma nu i beta j prime sigma beta i this summation is over i here this is equal to now scalar here this term will become scalar because I have multiplied by a rho vector on the left hand side. So, now, if lambda j is not equal to 0 then we will get nu j is to be 0 and of course, you can have the condition that lambda is equal to 0 then sigma beta i that is j that is equal to lambda j beta j that will be equal to 0 and j th term in 13 will vanish j th term will not be there. So, what we are ultimately getting finally, the same condition here that is sigma beta 4 must be satisfied and lambda must satisfy that means it is again the r plus 1th eigenvalue. So, let let me give a formal proof of this let lambda r plus 1 be the maximum of lambda 1 lambda 2 lambda p such that there is a vector satisfying sigma minus lambda r plus 1 i beta is equal to 0 beta prime beta is equal to 1 and the condition 11 that is this condition that lambda i beta prime beta is equal to 0 here. So, we can call this vector beta r plus 1 let us define the linear combination u r plus 1 as beta r plus 1 prime x ok. Now, if lambda r plus 1 is equal to 0 and lambda j is equal to 0 for j not equal to r plus 1 then the corresponding beta j prime sigma beta r plus 1 that is also 0 this does not imply that beta j prime beta r plus 1 is equal to 0. So, we can replace however we can replace beta r plus 1 by a linear combination of beta r plus 1 and those beta j's whose lambda j's are 0. So, again this new beta r plus 1 this is orthogonal to beta j for j is equal to 1 to r. So, we can continue this procedure this procedure is continued till m plus 1th stage such that we cannot find a vector beta which will satisfy 4 and 11 and beta prime beta is equal to 1. So, now either m is equal to p or m is less than p where beta 1 beta 2 beta m must be linearly independent. We can show actually that the m less than p is not possible it will lead to a contradiction because if m is less than p that means there will be p minus m vectors let us call them e m plus 1 e m plus 2 e p such that beta i prime e j will be equal to 0 and e i prime e j will be equal to something like delta i j which is the characteristic function that means it is equal to 1 if i is equal to j and it is equal to 0 if i is not equal to j. So, ultimately this can lead to a contradiction here. So, let me not discuss this full thing I will let me just give a brief hint here. So, this procedure is continued till m plus 1st stage and then one cannot find a vector beta satisfying beta prime beta is equal to 1 4 and 11. So, either m is equal to p or m is less than p as beta 1 and so on beta m must be linearly independent. We now show that m less than p is not possible. So, if m less than p is assumed then there exist p minus m vectors let me call them e 1 m plus 1 and so on up to e p such that beta i prime e j is equal to 0 and e i prime e j is equal to delta i j. So, let us consider the set e as e m plus 1 and so on e p. We show that there exist a p minus m component vector c and theta such that e c is equal to sigma c i e i is a solution of 4 with lambda is equal to theta. So, let us consider a root of say e prime sigma e minus theta i is equal to 0 and let us also consider a vector c which satisfies e prime sigma e c is equal to theta c that means, it is the Eigen value of this theta is the characteristic root and c is the characteristic vector here. So, this vector sigma e c this is orthogonal to beta 1 beta 2 beta n that means, it is in a span of e m plus 1 and so on up to e p that means, we can write as e g if it is in the span of this because e is the e m plus 1 up to e p that means, it is in the column space of that. So, you multiply by e prime. So, you get e prime sigma e c that is equal to e prime e g that is equal to g because what is e prime e? e prime e is the identity matrix. So, this means that g is equal to theta c and sigma e c that will be equal to theta e c is equal to theta that is this one. So, what we are getting that this is actually equal to 0 here. So, e c prime x is uncorrelated with beta j prime x for j is equal to 1 to n that means, there exist beta m plus 1 because we started with m less than p now I am saying I am there is an m plus 1. So, this is a contradiction. So, that means, m must be equal to p that means, our process of finding out the principal components is systematic that means, we are actually going up to that route. So, now let us consider this set now thus we have determined the set let us call it say script b that is equal to beta 1 beta 2 beta p these vectors we have obtained and the corresponding lambda I will call lambda 1 lambda 2 lambda p these equations sigma beta r is equal to lambda r beta r for r is equal to 1 to p can be written as sigma this b is equal to b lambda and this conditions that we obtained for normalization and orthogonality that can be expressed as this can be written as b prime b is equal to identity matrix. So, this 16 and 17 will then give me b prime sigma b is equal to lambda because I can multiply here b prime sigma b is equal to b prime b that is identity. So, that is equal to lambda now this determinant of lambda sigma minus lambda I we can write as b prime sigma minus lambda I b because b prime b is equal to I and I can split it on both the sides because determinant of say 2 matrix p q is equal to determinant p into determinant of q and that can be written as determinant of q into determinant of p. So, this can be split. So, this can be then written as b prime sigma b minus lambda I because b prime b is again I that is lambda minus lambda I this is product of lambda I minus lambda. So, the roots of 19 are diagonals elements of lambda that is we are getting lambda 1 is equal to lambda 1 and so on lambda p is equal to lambda p. So, we have determined the principal components. That means if I have a random vector x with expectation x is equal to 0 and the dispersion matrix as equal to sigma then there exist an orthogonal linear transformation u is equal to say beta prime x not beta let me call it b prime x such that dispersion matrix of u u prime is equal to dispersion matrix of u is equal to lambda and lambda is equal to lambda 1 lambda 2 lambda p this is all 0 0 where these lambdas are ordered these are roots of 5 and the r th column of b satisfies sigma minus lambda r I beta is equal to 0 this u r that is equal to beta r transpose x this has maximum variance among all linear combinations and uncorrelated with u 1 u 2 u r minus 1. So, this vector u that I have determined from x this is called the vector of principal components. So, in a practical problem what happens that you replace the original coordinate system by the new coordinate system which is u 1 u 2 u p and then you choose the ones which are relevant. That means, it may turn out that the variance of say u 1 is say 90 percent of the total variance variance of u 2 may be may be another 8 percent. So, you just look at the ones which cover the major portion and you can discard the remaining for your problems which are like finding out the relationships between the variable that means, you are setting up a regression model etcetera or any other thing that means, you are actually considering several variables, but then you can keep the relevant ones only you did not consider all of them. Now, the question comes that when I do not have the variance covariance matrix available to me in that case in the practical life we will have data set. So, we consider the maximum likelihood estimates of that for example, if I am dealing with the multivariate normal population. So, you have variance covariance matrix sigma. So, I can use S there that is the sample variance covariance matrix and I can use that to find out the principal components of that and from there I can determine. So, let me just mention this estimation problem also if the population dispersion is unknown then we can use estimation for finding principal components. Actually, let me consider say normal population let us consider say x 1 x 2 x n be a random sample from a p-variate normal population with mean vector say mu and dispersion sigma. I make an assumption that sigma has p different characteristic vectors well you know that why this condition is required this condition is for the diagonalization ok. So, then a set of MLEs of lambda 1 lambda 2 lambda p and beta 1 beta 2 beta p these are the roots k 1 k 2 k p of sigma hat minus k i is equal to 0 and a set of corresponding vectors b 1 b 2 b p satisfying sigma hat minus k i i b i is equal to 0 b i prime b i is equal to 1 this sigma hat is some MLE of sigma. If the roots of sigma are not of multiplicity 1 then this method will fail actually. Now, you may mention that if this method fails then what else we can do, but actually what happens that when you have a data dependent problem that means, when you have the observations then it is highly unlikely that any root will be repeated because the variance covariance matrix that you will be getting will be highly diversified actually. It will be positive semi definite, but the root repeated probability will be almost negligible here. Therefore, this condition is almost always satisfied here. Now, let me give an methodology how to determine this principal components because I have given a theoretical method, but in general if you have say dimension 5 dimension 10 because normally any problem will be of that nature only you cannot have a problem of 3 by 3 or 2 by 2. So, therefore, we need some efficient computational procedures by which we can determine this. So, we consider some methods for some methods for computation of principal components. So, if P is a small then we can consider this determinant of sigma minus lambda is equal to 0. So, it will be a equation well polynomial equation in lambda. So, you can apply some efficient method for example, Newton Raphson method or say secant method and so on. So, there are some efficient methods which can be used here. Now, if P is large that will be the usual thing that in practice you will have P large then you will look at sigma x is equal to lambda x ok. So, you consider some iterative methodologies for example, you may consider say x i is equal to sigma y i minus 1 then y i is equal to 1 by square root x i prime x i x i ok. So, this is for i is equal to 1 2 and so on then i is equal to 0 1 2 and so on. Then one can actually show that limit of this y i is will be equal to beta 1 and similarly limit of this x i prime x i that will be lambda 1 square ok. And basically the rate of convergence will be dependent upon the ratio lambda 2 by lambda 1 and closer this ratio is to 1 the slower the rate of the convergence. Then for the second one then again we consider sigma 2 minus lambda 1 beta 1 beta 1 prime and from there we determine. So, in general my comment is that one can use various numerical methods for determining the principal components here. I will close this by one example here. This example is discussed by Etkin in 1937. So, and of course, one can consider like q or algorithm, house holder algorithm, there are many algorithms for solving the system of equations. So, one can actually use this. So, let us consider the famous example of Fisher which is the having the data on the petal lengths, petal width, sepal length and sepal widths here. So, x 1 was sepal length, x 2 is the sepal width, x 3 is petal length and x 4 is the petal width. This data Fisher had considered for the certain trees iris verosa and something like that and for this the x i minus x bar x i minus x bar prime i is equal to 1 to 50. There are 50 data sets here and this was given by 13.0552, 4.1740, 8.9620, 2.7332, 4.825, 4.05, 2.019, 10.82, 3.582 and 1.9162 and these values are repeated here. This is a symmetric matrix here. This example is worked out in Anderson. So, sigma hat is actually equal to s that is 1 by 49 a. So, that can be calculated. If we consider say initial approximation as say 1010, then we get the value of l 1 as 0.487875 and the corresponding principal component is coming out to be 0.6867, 0.3053, 0.6237 and 0.215. This l 1 turns out to be this is l 1 let us call it. This is actually the largest root here coming out to be it is more than 3 times the sum of the other 3 roots. Similarly, if I find l 2 that is turning out to be 0.072 and the corresponding b 2 vector can be obtained and l 3 is 0.054, l 4 is 0.009. So, you can easily see the value of this is for exceeding these values here. So, this linear combination will be the principal component actually. You can say the first principal component which can be used. This is corresponding to basically more than 78 percent of the total variation. The last component is corresponding to less than 1 percent of the variation here. So, the ultimate aim of this principal component analysis is to determine that linear combination which is leading to the maximum variability or the which explains the maximum variability in the data. So, therefore, one should use that thing. Another related concept is that of canonical correlations or canonical variables. So, now in general we have some correlation between x 1, x 2, x p, but what we do we find out those linear combinations which will have more correlation among themselves ok. So, this type of thing will be leading to that means, we transform my x 1, x 2, x k to a new set say y 1, y 2, y k and these are arranged in the order of the correlation. So, this problem is a similar problem that is of the problem of principal components. In the principal component we are considering the variability. Here we are considering the correlation. So, for example, I consider say x as a vector say these are having p components then we split it into two parts. So, this is say p 1, this is p 2 and the corresponding sigma matrix is sigma 1, 1, sigma 1, 2, sigma 2, 1, sigma 2, 2. So, we develop a transformation of the first p 1 and last p 2 coordinates in such a way that the inter correlation between x 1 and x 2 will be correlated. That means, we find u is equal to alpha prime x 1 and v is equal to say beta prime x 2 such that this correlation u between u and v is maximum ok. So, this is the problem of canonical correlation. So, we can if we normalize the thing then it is same as correlation or covariance both will be the same and like the problem of this this can also be simplified and it turns out to be problem of eigenvalues of that this type minus lambda sigma 1, 1, sigma 1, 2, sigma 2, 1, minus lambda sigma 2, 2. So, this is a little variation from the previous problem, but it is similar in nature here. So, we find then this one then next we find the those linear combinations which are uncorrelated with the first set, but again having the maximum correlation. So, that way we can determine the canonical correlation. So, these are called canonical correlations. This problem has also been fully solved by hotelling and the solution is given. The problem has been applied to various real life data sets. These topics are also discussed in detail when we are not having the known variance covariance matrix. That means, we can have maximum likelihood estimates for that if we are dealing with the multivariate normal populations or if we are dealing with some other type of estimates when the form of the distribution may be something different. I wind up the multivariate analysis portion of this course at this stage. There are many more topics in multivariate analysis like factor analysis is there. So, but in this particular course we will not be touching upon that. In the next class I will take up a new topic.