 We will be continuing with our discussion on response surface methodology, we will be doing a new example, the previous example we saw how to construct a preliminary design based on 2 power 2 factorial with the centre points, we also checked whether the curvature effects would be important and we did not find evidence of curvature effects in the first example and then we also identified the path of the steepest ascent. In the second example we are going to find the optimum solution after going through the path of fastest progress and the path of fastest progress may be either steepest ascent or steepest descent depending upon the nature of the problem. So now let us look at the problem statement for the second example, so we are having an exploratory study conducted in a pilot plant using a CCD or central composite design strategy and the results are obtained as shown in the following table. So you are asked to find the second order model parameters, the sequential and adjusted sum of squares, the complete ANOVA table, the significance of the parameters and the nature of the response surface. So the data is presented here, it is a straight forward central composite design where you have the factorial points at the beginning, these are the responses and then you have the axial points given in the second set, this corresponds to the location of the alpha values – root 2 plus root 2, this is – root 2 plus root 2 corresponding to the axis for factor B. These are the responses and then you are also given the repeats at the center, you can see that there are 5 repeats, you have 4 plus 4, 8, 8 plus 1, 9 independent settings, the total number of experiments is 13, you have 9 independent settings because in addition to the 8 runs you are also having one independent setting corresponding to the center. So first we have to estimate the second order model parameters, there are a couple of ways of doing it, the simplest would be to go for the regression method, it is very convenient and straight forward especially in the matrix form and the regression method also helps you to quickly find the sequential sum of squares and the adjusted sum of squares. So the 6 parameters are found and are listed here, beta hat 0 is 65.25, it is the intercept estimate, beta hat 1 is the regression coefficient corresponding to factor 1 or factor A and that value is 1.1868 and beta hat 2 is – 0.49 which is the regression coefficient corresponding to factor 2 or factor B, beta hat 12 is only 0.165 and that corresponds to the interaction between the factors A and B, beta hat 11 is the regression coefficient corresponding to the pure quadratic term for factor 1, x1 squared, so square of the level of factor A. So beta hat 22 is the regression coefficient corresponding to the quadratic term involving the second factor given by factor B squared or x2 squared and the 2 values for the quadratic terms or the quadratic coefficients are 1.2912 and 0.7163. So we constructed the x matrix, the first column is the matrix of once and then you had corresponding to factor A, factor B, this would be the interaction between factors A and B and this would correspond to the quadratic term concerned with factor A and that is x1 squared, so you have all these numbers squared – root 2 squared would be 2, plus root 2 squared will also be 2. Similarly, this is for factor B, all the terms in the column are squared, so it would be a simple thing to take x prime x, x prime is the transpose of the x matrix I showed in the previous slide and you can see that the x prime x matrix is not a diagonal matrix, there are also certain terms in the off diagonals but you can also see that many terms are 0, just make a small modification to the slide. So you have the x prime x matrix and you can see that the off diagonal terms also exist but it is a sparse matrix because all the positions in the matrix are filled, there are many zeros in this matrix but the main thing to note here is it is not a purely diagonal matrix and when we take the x prime x inverse we get the result as shown here, the most important thing is we do not get a singular matrix and then you have x prime y and the values are given here and we can estimate the parameters beta hat as x prime x inverse x prime y and this is what we get, this is the intercept beta hat 1, beta hat 2, beta hat 1 2, beta hat 1 1, beta hat 2 2. So now we have to go to the next part of the exercise which is to find the sequential and adjusted sum of squares, I personally like finding out the sequential and adjusted sum of squares, it always fascinates me to see for the orthogonal design the sequential and adjusted sum of squares are the same, whereas for a non-orthogonal design the two are different and it shows the power of the orthogonality where the contribution of each factor and the interaction between the factors are clearly delineated and so one does not interfere with the other. In such a situation it is not really matter whether you bring the factor 1 first or factor 2 first or even the interaction between the two factors can be brought in first. So the sequence does not really matter whereas in a non-orthogonal design the sequence in which the model is built has its influence on the estimated model parameters. So again the regression approach is used to find the regression sum of squares, the concept is very simple in the sequential sum of squares what you do is you build a model as the name suggests sequentially beta hat0 plus beta hat1 x1 plus beta hat2 x2 plus beta hat12 x1 x2 plus beta hat11 x1 square plus beta hat22 x2 square. So this is the sequential method of model development and using the regression analysis the regression sum of squares may be computed and so you have a full model and then you have the model without a particular parameter being present the difference between the two regression sum of squares would give you the sequential sum of squares. So I would suggest that you revise the portions in linear regression so that what we are telling here would be immediately evident to you. As far as the adjusted sum of squares is concerned it is very interesting you take a particular parameter of interest to you and that particular parameter you introduce last in your model. For example if I am particularly interested in factor 2 what I would do is I will first develop a model involving the intercept the factor 1 and then the interaction between factors 1 and 2. So that would be my preliminary or original model and then I will add in the factor 2 at the very end. So I will have a preliminary model I will find the regression sum of squares and then I will also have the full model in which factor 2 is added last I will find the regression sum of squares. The difference between the two would give me the extra sum of squares or the adjusted extra sum of squares due to the second parameter. So the example I have chosen I have done all the calculations and given both the sequential and adjusted sum of squares I request you to do the problem on your own and see whether your results are matching with mine. So let us now look at the different model parameters that may be added to the gradually developed model. The model form after the addition of the parameter would be y hat is equal to beta hat not plus beta hat 1 x1 and this is the model and then the parameters are 66.4 and 1.187 corresponding to beta hat 1 and then when you add the second parameter the model is as shown here and that comes to 66.49 1.187 minus 0.49 beta hat 1 to the interaction term is added last after adding the intercept factor 1 and factor 2 and when you do that the estimated parameters are 66.484 1.1868 minus 0.49 0.165. It can be seen that when the model is being built the parameter beta 1, parameter beta 2 are unchanged in all these 3 models for example beta hat 1 is unchanged in all these 3 models and beta hat 2 is unchanged in the 2 models in second and third rows and then when you add the quadratic term corresponding to x1 squared we get the same values of beta hat 1, beta hat 2, beta hat 1, 2 and then you have the additional contribution from the quadratic term 1.1978 and then when you finally add the last quadratic term x2 squared the regression coefficient corresponding to that would be denoted by beta hat 2, 2 and that is what is present here in the model and that value is 0.7163 and when you add the last parameter beta hat 2, 2 it can be seen that beta hat 1, 1 changes. This is a very important difference here you did not have beta hat 2 but you had 1.187 for beta hat 1. Now when you added beta hat 2 parameter the value of beta hat 1 remained at 1.187 and when you added the interaction term the values of the main parameters beta hat 1 and beta hat 2 remained at 1.1868 and 1 minus 0.49 and then you added 0.165 as the interaction term but when you do the same thing for the quadratic terms you find that beta hat 1, 1 initially was 1.1978 when it was alone present in the model and when you added the beta hat 2, 2 the value of beta hat 1, 1 changed from 1.1978 to 1.2912. On the other hand the main factors in the interaction factor A factor B and factor AB interaction remained at 1.1868 minus 0.49 and 0.165 this is very interesting because the model is having the factorial contribution and which is orthogonal in nature and that is why the main factors and the interactions did not change even though the model was gradually developed. On the other hand the non-orthogonal component was created by the quadratic terms and so the values changed when additional parameters were included in the model. So the sequential sum of squares for each model is listed in this table. So the sum of squares due to beta hat 1 is 11.267 and the sum of extra sum of squares due to sequential addition of the second factor is given by 1.9207 and then for the interaction the sequential sum of squares is 0.1089 and for the quadratic term addition the extra sum of squares is 10.1539 and then you can see that beta hat 2, 2 is adding 3.5688. So by looking at the sum of squares addition we can also gauge the relative importance of the particular parameter. So you may expect that when compared to beta hat 2, beta hat 1 is quite or having more effect whereas the extra sum of squares brought in by beta hat 1, 2 is only 0.11 and this 0.11 and so you may expect the contribution from beta hat 1, 2 to be small. Between the 2 quadratic terms you would expect beta hat 1, 1, 2 have a greater say in the model response because the extra sum of squares due to beta hat 1, 1 is about 3 times more than the extra sum of squares brought in by beta hat 2, 2. So to calculate the sequential sum of squares you use the regression approach and then you take the fully developed model regression sum of squares and subtract from it the regression sum of squares for the model which did not have that particular parameter. I request you to carry out the calculations yourself and check whether the results are correct. The calculations are not very difficult especially if you do it with some standard software like MATLAB or SyLAB and next we move on to the adjusted sum of squares calculation and here the important thing is we are adding the particular parameter of interest last after having added all the other parameters. So you can see that beta hat 1, x1 is added last. The important thing to note here is the coefficient values are the same as before. This is the intercept. The intercept of course is not changing and in this case and you can see that factor 1, factor 2, factor, this is factor 2, this is the interaction between first factor and second factor. This is the quadratic contribution from factor 1 and quadratic contribution from factor 2 and you can see that it is 1.1868 for beta hat 1 and when you want to add beta hat 2 the last it is minus 0.4 and which is the same value that was present in the earlier models as well and when you had the interaction term the last you have 0.1650. Other parameters are the same and when you had the quadratic term beta hat 1, 1 the last you get 1.2912 and then beta hat 2, 2 you get 0.7163. So the adjusted sum of squares are given in the table below and you can see for the orthogonal components involving the main factors in the interaction the adjusted and the sequential sum of squares are the same. The sequential sum of squares are given in the brackets next to the adjusted sum of squares. You can see that the adjusted and sequential sum of squares are the same for the 2 main factors and the interaction between them and in this slide you can see that the quadratic term beta hat 1, 1 the adjusted sum of squares is different from the sequential sum of squares. Since the parameter beta hat 2, 2 is added the last the adjusted and sequential sum of squares for this particular parameter is the same. So to summarize some of the important points the coefficients in the sequential as well as the adjusted sum of squares were obtained very quickly and rapidly from the regression technique. Extra sum of squares principle was used to obtain the sequential and adjusted sum of squares. Okay some more additional points to note are the fact that factors a, b and a, b could be seen as members of the elite club the orthogonal design club and their values did not really change when they were estimated in a different order. On the other hand the beta hat 1, 1 and beta hat 2, 2 were seen to be dependent on one another. For example beta hat 1, 1 took a certain value when it was the only parameter that was being estimated and when beta hat 2, 2 was included in the model the value of beta hat 1, 1 also got changed. Whereas irrespective of whether the orthogonal design parameters were there or not for example beta hat 1 did not really depend upon the presence or absence of beta hat 2 and another interesting thing is irrespective of whether beta hat 1, 1 and beta hat 2, 2 were included in the model or not the parameters beta hat 1, beta hat 2 and beta hat 1, 2 took upon the same values. So these are some finer points to understand in these kind of designs and we also found for the orthogonal components the adjusted and sequential sum of squares were the same whereas for the non orthogonal components the sequential and adjusted sum of squares were different. When we report the squares and apply them in the ANOVA table we use the adjusted sum of squares. Minitab uses the adjusted sum of squares in the analysis of variance table and this could be important. For orthogonal designs it does not really matter because the sequential and adjusted sum of squares values are the same. However when you go for the non orthogonal component of the design or even completely non orthogonal designs sequential and adjusted sum of squares can be different and we have to be clear on which one we used for our F value calculations. So here we are looking at the adjusted sum of squares. So finally we have the ANOVA table and you can see that the values are given here I will just check one thing right. So even though the sequential sum of squares are written here we are using the adjusted mean squares. For AB and AB does not really matter because the adjusted and sequential sum of squares are the same. However for A square even though the sequential sum of squares is 10.1539 the adjusted sum of squares is 11.5988 and when you divide by 1 degree of freedom the adjusted mean squares comes to 11.5988. So for B square since it is the last parameter being added the sequential and adjusted sum of square values are the same and you get the adjusted mean squares as 3.5688. Please remember an important issue here. Here the pure errors is based on the centre points whereas the residual sum of squares is obtained from the formula y prime y-beta hat prime x prime y that is the sum of squares not accounted by the regression model. So y prime y is the total sum of squares and beta hat prime x prime y is the regression sum of squares and the difference between the 2 gives you the residual sum of squares. And as we know by now the residual sum of squares is split into lack of its sum of squares and the pure error sum of squares. The pure error sum of squares is obtained from the centre points. So here we are talking about the residual sum of squares and the degrees of freedom is an interesting concept here. So that is given as n-p where n is the total number of experimental settings which is 13 in number. So you have 4 plus 4, 8 for the factorial points and the axial points and then you have 5 repeats that is 13 runs and then you have 6 parameters which we saw 1, 2, 3, 4, 5 and then the intercept beta hat 0. So that makes it 6 parameters and so you have n-p as 7 and that contributes to the residual degrees of freedom. So we divide the residual sum of squares by the degrees of freedom for the residual sum of squares and we get 0.2267. The ratio of the adjusted mean squares with the error mean square gives you the f value and you can see that the f values are reported in the second last column and these values are pretty high indicating that many of the parameters are significant. AB is very small, f0 values also 0.4 and the corresponding p values have been also computed and alpha is equal to 0.05 that is the level of significance and we can see that except for the interaction between A and B all other terms are significant. We take a closer look at the residual sum of squares and it is composed of lack of its sum of squares and pure error sum of squares and here I will just make a small correction. I think this should be capital N. So please note that correction here we have the residual sum of squares or the error of sum of squares and that is given by 7 degrees of freedom total number of exponential runs is 13 which is capital N, p is the number of parameters which is 6 and so we have 13-6 is equal to 7. Taking a closer look at the residual sum of squares it is split into lack of its sum of squares and the pure error sum of squares. So we have 7 degrees of freedom for the residual sum of squares and that is split into 3 degrees of freedom for lack of it and 4 degrees of freedom for pure error. You can easily find the degrees of freedom for the pure error because we had 5 center points, 5 repeats at the center and so we have 4 degrees of freedom for the center and since the residual sum of squares is having 7 degrees of freedom the remaining 3 degrees of freedom is assigned to the lack of fit and so we can calculate the mean squares here and another important thing we do here is we want to see whether the lack of fit is significant and for that purpose we take the ratio of lack of it sum of squares to the pure error sum of squares and then we see that the lack of it sum of squares is insignificant because the P value is quite high. The F value is obtained by taking the ratio of 0.1124 for lack of it sum of squares 2.3125 the pure error sum of squares. So the significance of the parameters we found from the ANOVA table except for the interaction term all other parameters were significant the main effects A and B and the quadratic terms corresponding to A squared and B squared were all found to be significant and we also found the model we have developed is adequate because the lack of fit turned out to be insignificant and so we have to see the nature of the response surface we have to identify the optimum point and characterize the optimum point. So MINITAB provides nice pictures 3D plots this is the response and this is the factor A and factor B you can see that there is a nice depression at the center so we have to do some mathematical analysis to confirm that we have really hit upon the optimum minimum solution. How do we go and do that? First let us now look at the degrees of freedom analysis we have 13 runs 4 factorial runs 4 axial runs and 5 repeat points and so we have 13 points and or 13 settings and out of which we have determined 6 parameters and so we have 7 degrees of freedom and the number of independent settings for our run are 4 factorial conditions which are all independent of each other 4 axial points which are all independent of each other and the 5 center points correspond to only one independent setting because you are repeating the 5 data points you have only one independent setting and so we have totally 9 independent settings and we have estimated 6 parameters and so the lack of fit pass would be 3 so this is another way of looking at it the 6 parameters we estimated are including the intercept beta hat naught beta hat 1 beta hat 2 beta hat 1 1 beta hat 2 2 and beta hat 1 2 so we are having 6 parameters estimated. Now going on to the location of the stationary point we have the full model given here y hat is equal to beta hat naught plus beta hat 1 x 1 plus beta hat 2 x 2 plus beta hat 1 2 x psi x j which in our case is beta hat 1 2 x 1 x 2 I just make that correction okay so now we have the next task of finding the stationary point and identify the nature of the stationary coordinates how the surface is shaped or behaving at the stationary point identified. So this is the model we are going to work with this is the second order model as we have been discussing so far. So to find the stationary point coordinate we need to take partial derivatives of this y hat with respect to x 1 and then with respect to x 2 and we set the partial derivatives to 0 and we solve for the x 1 and x 2 to identify the stationary point coordinate that is one way of doing it and by just locating the stationary point we cannot really say whether we have reached the maximum or minimum there is another elegant way of doing it and that would be the matrix approach and so given the second order model we can construct the matrices as shown in the following slides we can represent the second order model which is given here in a matrix form where y hat in the vector containing all the predicted values of y is expressed as a sum of beta hat naught which is a scalar plus x prime b plus x prime b x where x is the usual x matrix and we are having 2 new matrices here small b column vector and capital B matrix let us see the shape of these 2 matrices the general model may be expressed what is the model we are having a second order model as given here and we express this model in matrix notation in the following way y hat is equal to beta hat naught plus x prime b plus x prime b x I will define the different matrices in the next slide y hat is the column vector of the predicted responses beta hat naught is a scalar quantity what are x and small b and capital B matrices please note that this x matrix is different from the earlier x matrix we applied in linear regression. So, here we have x is equal to x1 x2 so on to xk the independent settings the factors this can be factor A factor B next you can have the interactions and then you can have the quadratic terms and so on and then you also have the B matrix which is given by beta hat 11 beta hat 22 and beta hat kk along the diagonals so the quadratic terms are present along the diagonals this is very interesting the off diagonal terms are interesting these are the interaction between the different factors divided by 2 because you have beta hat 12 and you can also have beta hat 21 so beta hat 12 represents the interaction between factor A and factor B or factor 1 and factor 2 and the beta hat 21 would also represent the interaction between factors 211 so interaction between factors 1 and 2 will be equal to the interaction between factors 211. So, you can say that the capital B matrix is a symmetric matrix where the beta hat 12 is equal to beta hat 21 and so that contribution is then split as beta hat 12 by 2 and then when you have the second row first column that would be again beta hat 12 by 2 and similarly if this is the beta hat 1k by 2 then the last element here would also be beta hat 1k divided by 2 so these correspond to the interaction terms the off diagonal terms in the capital B matrix refer to one half of the interaction parameters and then the B matrix on the other hand is pretty straight forward these are the regression coefficients of the main factors or associated with the main factors so B is a symmetric matrix as I told for in the previous slide of order k by k remember we have in addition to the intercept beta hat not we have k regression parameters and B is a symmetric matrix of order k by k whose main diagonal elements of the pure quadratic coefficients beta hat ii and the off diagonal elements are one half the mixed quadratic coefficients beta hat ij the interaction terms where i is not equal to j and the vector B is having dimensions of k rows and one column it is a vector of first order regression coefficients corresponding to the main factors and the dimensions for B as I said this k by k now we are having the second order model represented in the matrix form as shown here we also saw this form in one of the earlier slides so we differentiate y hat with respect to x and we get the following equation in matrix form B plus 2 B x and when you solve for x you get minus half B inverse B and this directly gives us the stationary point coordinate and the predicted value at this stationary point is y hat s is equal to beta hat not plus half of xs prime into B where xs is obtained from this equation. So for the present problem we have beta hat we saw this earlier so these are the values of the beta hat vector and you can see that these are the main factors corresponding to the B matrix and beta hat 1 2 by 2 would be the off diagonal term in the capital B matrix and beta hat 1 1 and beta hat 2 2 would be the diagonal terms in the capital B matrix and so that is what you have 1.2912.7163 are the quadratic terms they are present as such and then you have beta hat 1 2 as 0.165 and so when you take half of that it becomes 0.0825 in the 2 off diagonal locations and then B you have the main factors 1.1868 and minus 0.49 and xs is equal to minus half of B inverse of B. So I will just make a small correction here so we can easily find the location of the stationary point by taking minus half of B inverse of B we take B inverse here B is this matrix and then you have B the main factors 1.1868 minus 0.49 1.1868 minus 0.49 so when we take inverse of this and then multiply with B and then you take the resulting product at divided by minus 2 we get the stationary point as minus 0.4850 0.3979 so this is very interesting you could have done it the other way also you could have taken the partial derivatives of the response equation and set it to 0 and then solve for it to get the 2 values they would have been exactly same as minus 0.4850 and 0.3979 so in order to identify whether the optimum we have obtained this maximum or a minimum we have to do a bit more analysis we take the eigenvalue of the B matrix and the eigenvalue is definition goes as follows it is a determinant of B minus lambda I where I is the identity matrix of the same order as the B matrix. So when you do that you find you get the following determinant 1.2912 minus lambda 0.0825 0.7163 minus lambda and so we said this is equal to 0 and when we solve for it we get lambda 1 as 0.7047 lambda 2 is 1.3028 both the eigenvalues are positive which means that the obtained solution is a minimum and that is what the response surface also seem to indicate. So we come to the conclusion of the response surface methodology approach by using 2 examples we have seen how to handle central composite design structures how to find the direction of steepest ascent how to identify the optimum point and then characterize the optimum point. A similar approach can also be done using the box Benken design there are several problems available in the standard textbooks I would recommend you to use a regression approach to estimate the parameters for the different design strategies we have covered so far. So this is an important chapter of tremendous implications in the industry I request you to actually do experiments using this approach and you can see the power of this approach. So thanks for your attention.