 Welcome back, in today's lecture we will be looking at some experimental design strategies. The references for this lecture are the book written by Mayors Montgomery Anderson Cook Response Surface Methodology, Process and Product Optimization Using Designed Experiments 3rd edition, John Wiley and Sons, New York 2009. You may also want to refer to Montgomery Design and Analysis of Experiments 7th edition, John Wiley and Sons, New York. The importance of central composite design will be stressed upon in this lecture. It is a very popular second order design used widely in both research and in industry. Three levels are employed and there are some features like rotatability and good prediction variance properties. The central composite design enables the development of second order model and incorporates curvature. What is meant by a second order model? So far we have been looking at the main factors and the interaction between the factors. When you want to expand in the model space, the response may show curvature and in a multi-dimensional coordinate system, you will have the response surface in the form of a 3-dimensional surface and to describe such kind of response surfaces, we need higher order terms in the model equation. Second order terms like x1 squared, x2 squared, usually we do not go for models higher than second order unless it is absolutely essential. So let us see how we may develop the second order model using the central composite design approach. This is also used in response surface methodology designs when searching for the optimum. So why do we emphasize so much on second order models? The experimental design space, the response surface is no longer planar but may be marked by peaks and or valleys. Second order models are required to estimate this response and enable the identification of optimum solution if any. Second order models are of the form y is equal to beta not plus i equals 1 to k beta i xi sigma i sigma j beta i j xi xj plus sigma i equals 1 to k beta i i xi squared and so on. So this is the main factors and this is the interaction, binary interaction between two factors taken at a time and then you have the second order terms x1 squared x2 squared and so on. This is of course the error term. So when you want to fit a model, you fit one intercept and k main factors kc2 binary interactions and k second order terms. So that would be 1 plus 2 k plus k into k minus 1 by 2 parameters. If k is equal to 2, you will have 1 plus 4 5 and then 2 by 2 is 1 that would be 6 parameters totally you have to estimate. So those would be beta not, beta 1, beta 2 that makes it 3 and then one interaction term x1 x2 that makes it 4 and then x1 squared and x2 squared coefficients that makes it 6. So when we go for central composite designs, we are no longer able to retain the orthogonal property and we shift our attention from the orthogonal property advantages to the suitable low values of the scaled prediction variance. So when we develop a second order model, we are very worried about its prediction capability and we want to make the variance in the predicted response as low as possible. So what would be the suitable design strategy which will bring down the scaled prediction variance is our goal. What is the structure of the central composite design? So we add a central and 2 k axial or star points to a 2 factorial design. So let us take a simple case of a 2 factorial design. First we add center points. We have already seen the center points at the geometric center of the design space. They were used to not only get an idea about the experimental error but also regarding the significance of the curvature in the response. Then on top of these center points, we also add points along the axis. For a 2 factorial design, we have 2 axis, the x and y axis or x1 and x2 axis and you put certain points at select locations on the axis. On each axis, you put one pair of points symmetrically. So when each axis contains one pair of points for a 2-factor design involving 2-axis x1 and x2, we will have 4 axial points totally or 2 pairs of axial points. So the design compresses of 2 power k factorial points, the ones which are located at –1 plus 1 and so on, NC center points and 2k axial points. So the center points enable the identification of curvature in the system. If curvature evidence is irrefutable from a t-test or a suitable test, the axial points enable the efficient identification of the pure quadratic terms. So each point in the central composite design has its own significance. So what is the central composite design when looking at it pictorially? You have a central composite design shown here for 2 factors. These are the points in the experimental space. As usual, the 2 power 2 factorial design has 4 corner points each located at –1 and plus 1. So this would be –1, 1, 1, 1, 1-1 and –1, –1. The usual factorial design and this is the center points. You can have more than one center point. And then what is unique about the central composite design from the regular factorial design is the presence of the axis or star points. You can see that each axis, this is the x1 axis is having 2 points located at 1.414 and –1.414. Similarly, you have 2 points located on the y-axis or x2 axis and they are also located at 0, 1.414 and 0, –1.414. So the factorial points belong to the orthogonal and variance optimal class of designs. And these enable the identification of the main effects. The factorial design we saw comprised of points which were located on the extremes of the design space for that particular design –1 and plus 1. And since the points were located at very far off positions, you can visualize that the x prime x inverse matrix would be, x prime x inverse matrix would be pretty small and that would reduce the variance of the predictions and hence it was termed as an variance optimal design. The factorial points are used to find the main effects and the interactions. You find the main effects and the interactions in exactly the same way as you did for the regular factorial design. The center points also enable the estimation of the pure error as they represent repeats. So you need at least 2 or more repeat points and rather than repeating the experiments at all the factorial points, you may want to do the repeats at the geometric center. By this way you can get an idea about the experimental error and also you can save time on doing the experiments at the corners of the factorial design. Of course that would lead to more number of runs but in some cases that may be inevitable for the simple reason that certain research requirements require the reporting of the experimental measurements averaged over triplicates. So the center points in other cases are helpful to find the experimental error but in addition to this they also have another utility. The central points also help in the detection of the second order or curvature effects but do not help in their explicit individual estimation. Center points also give us a hint on whether curvature effects are important or not and they tell that whether the curvature is significant but it does not help us to explicitly quantify the curvature. It only indicates whether curvature should be considered in the model or not. So in order to identify the curvature effects explicitly we require the axial points. Why should the axial points be located at minus root 2 or minus 1.414 and plus root 2 or plus 1.414? The answer to this would be given shortly. So the axial points contribute to the estimation of the individual pure quadratic effects significance and if the axial points were not present only the sum of the quadratic term significance could be gauged using the center points. And the axial points do not contribute to the estimation of the interaction effects. The central points and the axial points contribute to the flexibility of the central composite design. So by adding the new central points and axial points which are variance or enhancements to the regular factorial design we make the experimental design more flexible. And so where do we exactly locate the axial points is the next question. It depends upon the region of interest in the experimental space and the number of central points determine the distribution of the scaled prediction variance in the region of interest. So the location of the axial points depend on the region of interest in the experimental space and the number of central points determine the distribution of a scaled prediction variance in the region of interest. This is a very important statement because we want to have our model predict uniformly as much as possible in the entire design space. If the variability in the prediction is unmanageably high in our design space then the models utility is reduced. It is not enough if the model predicts well in the center of the region the center of the geometric design space, the geometric center of the design space but also as we move away from it as we approach the edges of the design space we want the variability in the predictions to be kept as low as minimum. Because we normally want to predict the response of the experiment at points further and further away from the geometric center. We may want to even extrapolate sometimes the experimental response beyond the factorial points. In such cases if the variance in the predictions keep increasing then the utility of the model is lost. So planning for this we should see what should be the appropriate design strategy and we should also consider parameters like number of center points that would reduce or minimize the scaled prediction variance. And an important thing to note here is when you are planning the design strategy you do not need the experimental data explicitly. You can find the scaled prediction variance even before you carry out the experiments and see whether for the experimental strategy you have adopted the scaled prediction variance is manageable and is acceptable. So what we do is we add central and 2k axial or star points to a 2 power k factorial design. Suppose you have a central composite design with 3 factors then you locate the axial points at plus or minus alpha 0, 0, 0 plus or minus alpha, 0 and 0, 0, plus or minus alpha. How to determine the alpha is an important question we will answer it shortly. So the design comprises of 2 power k regular factorial points, NC center points and 2k axial points. So let us look at the MINITAB output for a central composite design involving 3 factors. You can see the 3 factors are represented by x1, x2, x3 here and then the first 8 experiments are the regular factorial design points. You can see minus 1, minus 1, minus 1, 1, minus 1, minus 1, so on. Then the 8th one is 1, 1, 1. Then you have these axial points minus alpha 0, 0, plus alpha 0, 0, minus alpha 0, 0, plus alpha 0, 0, 0, minus alpha 0, 0, 0, plus alpha, where alpha is 1.68179. What is a special magic number? We have to see shortly. And then you have as many as 6 repeated points at the geometric center of the design. So let us estimate the model coefficients which are in associated with data that are in coded units. We have to estimate the model coefficients for the experimental data that are in coded units. So we have to estimate the intercept main factors, binary interactions and quadratic effects only. So in addition to the intercept beta 0, we need to estimate the main factors coefficients. The coefficients associated with x1, x2 and x3. Then we have to identify the binary coefficients associated with x1, x2, x1, x3, x2, x3. And then we also have to find the coefficients associated with x1 squared, x2 squared and x3 squared. So the total number of design points is 20 for a 3 factor design with 6 repeats. So you can see that there are 20 independent experimental settings. That is not correct. It is not 20 independent experimental settings. You have 14 independent experimental settings and then even though you have 6 repeats that will constitute only one independent experimental setting. So that would mean 14 plus 1, 15 independent experimental settings are there. So you have total number of design points as 20. The degrees of freedom for model is 19 excluding the intercept and total number of regression coefficients estimated each with the degree of freedom is 3 plus 3 plus 3 that is equal to 9. The remaining degrees of freedom is 10. So the number of center points is 6. Lack of degrees of freedom for pure error is 6 minus 1 which is equal to 5. So lack of it degrees of freedom is equal to 5. So this is a very interesting calculation for the degrees of freedom for lack of fit. So even though we have fitted 1, 4, 4 plus 3, 7, 7 plus 3, 10 parameters the model possibilities are not exhausted. So there is still some scope for expanding the model and adding more coefficients. What can be the number of coefficients that can be further added to the model has to be first estimated. So if you look at the model you are having 20 experimental settings but out of that you are having 14 central composite design points, the factorial points and the axial points that would be 8 plus 6 because you have for 3 factors, 2 factorial design you are having 8 factorial points and for 3 axis you are having 6 axial points. So that makes it 8 plus 6, 14 and then you are having 6 center points. But the center points are repeats only that means that would constitute only one independent data setting. So in total we have something like 14 plus 1 which is 15 independent experimental settings and if we have already estimated 10 parameters and there are 15 independent experimental settings we can quickly say that we can additionally estimate 5 more parameters to the model that may not be really necessary but it gives us the option of adding another 5 parameters to the model because of process knowledge and prior experience there may be some unusual terms like x1, x2 squared or x2 squared, x3 these kind of terms may have to be added to the model because of the peculiarities of the process and then you may need to identify the coefficients associated with those variable combinations. Hence we have 5 more degrees of freedom for fitting additional model parameters and this is nothing but the lack of fit degrees of freedom. Sometimes even with 10 parameters there may be scope for model development and so the analysis of variance table would indicate that the lack of fit degrees of freedom is significant and hence we may have to consider adding of more terms to the model. So the lack of fit degrees of freedom is 5 as we discussed just now. So we can fit additionally 5 more regression coefficients after expanding the model appropriately without the risk of aliasing and now the distribution of experimental design points has a profound influence on the scaled prediction variance okay. So recall that the model developed is expected to fit the experimental data properly in the design space the SPV is a measure of how well the data is fitted by the model. So these concepts are very interesting for the simple reason that these are over and above what we usually are aware of in experimental design. There are numerous instances of siting of central composite designs in research papers and they get the justification that they are being mainly meant for considering the second order terms in the model. But many of these papers do not discuss further as to why the central composite design was chosen among different options available and how good is the prediction capability of the model developed using the central composite design. So these are probably beyond the scope of the particular research article but it is very important for us as data analysts and researchers to assess the quality of the developed model. How good the model is and it is also good to be informed about the limitations of the model in the design space. One important indicator of the limitation of the model in the experimental design space is the scaled prediction variance and that is the reason why we are harping on it for so many slides. In some cases the model may get frayed at the edges so that the scaled prediction variance may be very high at the boundaries. Even though the scaled prediction variance may look manageable in the interior portion of the experimental design space as we go further towards the extremes or the boundaries of the experimental design space the scaled prediction variance may shoot up very alarmingly and then the model is not very good at the edges of the design space. In certain cases there may be problems even at the center of the experimental design space the scaled prediction variance may be high at the center of the design space as well and hence to keep it down or control it we need to increase the number of center points in certain designs. So the scaled prediction variance if you recollect is given by spv of x is equal to n which is the size of the experimental run. The total number of runs in the experiment is n variance of y hat x divided by sigma squared and by doing so we are making the prediction variance independent of sigma squared which we do not know anyway. So we are also getting rid of sigma squared and we are also scaling the design for the size. Certain designs which are having large number of observations may artificially bring down the prediction variance because of the large size of the runs to account for that or to normalize for this effect we are multiplying by the term n. As an example if an experiment is performed with large number of repeats let us say 25 experiments have been performed with large number of repeats the prediction variance in such a case would be lower than another experiment where the number of runs was only restricted to 20. So to compensate or account for the size of the run we multiplied by n and so the prediction variance which is multiplied by n and then divided by sigma squared is termed as the scaled prediction variance and we have already seen how to determine the scaled prediction variance. We use x prime x inverse matrix and we also take the coordinate at which we want to estimate the scaled prediction variance and expand it to model space as was discussed in one of the previous lectures. We introduce at this point the moment matrix m which is defined as m is equal to x prime x by n. We saw that the variance covariance matrix is given by x prime x inverse sigma squared. So the x prime x inverse or the x prime x matrix is a very very important term because it captures the essence of your experimental design. Whatever design strategy you are implementing is present in the x prime x matrix and the inversion of that matrix help us not only to determine the coefficients of the model proposed but also the variability in the model coefficients and also the variability in the process response. So these are very significant in experimental design analysis, experimental data analysis and linear regression and in such a context the x prime x matrix assumes the center stage. So what is the moment matrix? It is x prime x divided by n. For a first order factorial design of order k with that means k parameters, the moment matrix is identified with an identity matrix of size k by k. Suppose you are having the order as k, the identity matrix would be having order of k by k. So let us now look at the second order models more closely. We define the moment matrix m as x prime x by n. For a first order factorial design of order p, the moment matrix is the identity matrix of size or dimensions p by p. We recollect that p is equal to k plus 1 where k is the number of regression coefficients beta hat 1, beta hat 2 and so on to beta hat k in addition to the intercept beta hat 0. So we are having p is equal to k plus 1 regression coefficients. So the x prime x matrix will also have dimensions of p by p and the moment matrix x prime x by n would be an identity matrix. The x prime x matrix for a first order factorial design would be a diagonal matrix and when you scale this diagonal matrix by the total number of runs, we get an identity matrix of dimension p by p. Let us demonstrate it here and we are having the x matrix which is given by 1, a, b, c, a, b, c, a, c the 3 binary interactions and then you have the ternary interaction a, b, c. So this is the x matrix and this is the column of ones and this is the column containing minus 1, 1, minus 1, 1, minus 1, minus 1, 1, 1 and so on. So we have the entire x matrix. To generate a, b, we just simply multiply the elements of the a and b column vectors similarly for b, c and a, c and so on and then you also have a, b, c which is 1 because it is minus 1 into minus 1 which is plus 1, 1 into 1 is 1 and so when we do m is equal to x prime x by n, we take the transpose of the x matrix and we then multiply with the x matrix again and then divide it by the number of settings n in this case is equal to 8, 1, 2, 3, 4, 5, 6, 7, 8. And when we do that the x prime x matrix would be diagonal matrix having 8, 8, 8 and all that but when you divide it by 8 then it becomes an identity matrix of dimension 8 by 8. So very interesting. So now let us define the different moments. You have the first moments represented by i and that is given by 1 by n sigma u is equal to 1 up to n xi u, u is the index for incrementing from 1 to n and i refers to the ith column or the ith model parameter. For example if you look at this particular column if we are talking about x1 u then we take this column corresponding to all ones in the first index and then u is running from 1 to n. So we go from x11, x12 so on to x1n. So the simple thing to note here is we are referring to the ith column and summing over the elements present in the ith column. So and that summation is carried over all the experimental settings in the data set and when you look at the second pure moments we have no adulteration of i with j and vice versa, i is present with the i and since it is present as a couple it is a second pure moment. And how do we find that? We take the square of the column elements we are choosing. Suppose we have chosen bracket i close bracket corresponding to the third column then it would be we will go to the third column in this x matrix and then we will do x31 squared, x32 squared so on to x3n squared in this matrix if i were to be 3. When you have second mixed moments the column vectors we are considering are different from each other. We are considering 2 column vectors and in these 2 column vectors i and j are different and so we multiply the individual corresponding elements in each column vector so that we get second mixed moment this i and j, i is not equal to j, i and j are different and hence it is called as mixed moment and that we do 1 by n sigma u is equal to 1 to n xi u into xj u. So we also have the third pure moment i, i and i which is since it is pure there is no additional or different term in the moment consideration it is i, i and i that means 1 by n sigma u is equal to 1 to n x cube i u. So for the 8th column vector we just take the cube of each element in that particular column vector and then sum it up over all the experimental settings. And similarly we can have all these other moments also third mixed moment the total order of the moment is 1 plus 1 plus 1 which is 3 and mixed moments means there can be elements which are different from one another you can have 2 same elements and then you can have a different element j. So this i's and j's obviously refer to the 8th column on the jth column in the moment matrix. So we have 1 by n sigma u is equal to 1 to n x squared i u and xj u okay a correction at this point these do not refer to elements in the moment matrix they are referring to the elements and column vectors in the x matrix okay we use the elements in the x matrix we use the column vectors in the x matrix to find the different moments. And if you look at the third mixed moment ij k it is 1 by n sigma u is equal to 1 to n x i u xj u xk u. Fourth pure moments are also possible where we take the fourth power of the elements in the 8th column vector and then submit over the n experimental settings. Fourth mixed moments would be the presence of different elements even i squared j squared is considered as a fourth mixed moment even though you are having 2 of a species or 2 of certain type together and that is let us say i is present together and jj is present together but since i and j are different we term it as a fourth order mixed moment and that would be given by 1 by n sigma u is equal to 1 to n x squared i u x squared j u. So you can also have ij k l where all the elements within the brackets are different so they obviously refer to different columns the ith column jth column kth column and lth column in the x matrix and all the i's j k and l are different from one another. Here we have the case of i squared j k the elements again are being different from each other and hence it is called as a mixed moment. So for a first order design a factorial design the first moment for any i is 0 so when you are looking at the x matrix if you look at any column we are not having columns with the contributions from x i squared like x 1 squared or x 2 squared. So all the elements in the x matrix for this case would be comprising of minus 1 plus 1 and so on except for the vector of 1's all other columns would be having minus 1 and plus 1 and when you total it up for each column it will become 0. For example if you look at the main effects x 1 or x 2 or x 3 each column would be having minus 1 and plus 1 in equal number and so when you take the sum it will go to 0 and that is what is meant by the first moment for any i is equal to 0 for the first order design and the second pure moment is unity. You may ask how it is possible the second pure moment is either i squared or j squared and so each of the minus 1 or plus 1 will uniformly become plus 1 only after squaring. So when you are having let us say 8 runs you are going to have the sum as 8 but please remember according to the definition of the second moment we are or for that matter any moment we are dividing by n. So that 8 will get cancelled with the 8 and hence you will get 1. So you can see the second pure moment is having 1 by n here sigma u is equal to 1 to n x i squared x squared i u and so all these things would be once and you are adding it up to n times means you will get n and n by n would be equal to 1. The second pure moment is unity that is what we saw just now because we are dividing all the squared elements with the size of the run and so they cancel out and the resulting answer is just 1. The first moment is analogous to the sample mean and the second moment is analogous to the sample variance and this mixed moment is analogous to the sample covariance. For a first order design the moments are up to order 2 and for a first order design the first and second mixed moments also called as odd moments at least one variable with odd power are 0. So the odd moments are 0 for a first order design. The second pure moment called as even moments are equal to 1 for the first order design. So now let us look at a saturated 2 power 3 factorial design x matrix. So you are having typical pure and mixed moments for a 2 power 3 design x matrix and so you see that the x matrix is having the usual column of once. It is having the column of A, B, C the main factors A, B, C, A, C the interactions A, B, C which is the ternary interaction and then you also have B into A, B, C squared. So all these things are created very easily. For example the column B, AB or B squared A is created by squaring B squared sorry by squaring B. So these values will all become 1 and then multiplying with A. So when you get B squared A 1 into – 1 will be – 1 and similarly B squared will be 1 and A would be 1 and so you are having – 1. Similarly you can find out C squared and B squared A would be a third order moment, third order mixed moment because A is also present here and when you look at the first order moments corresponding to the main factors when I am totaling all these values it becomes 0 and if I am even looking at B squared AB it is also equal to 0 because it is having equal number of –'s and plus here but when I do C squared it becomes 1 throughout the column and when I add it up 1, 2, 3, 4, 5, 6, 7, 8 it becomes 8 and the size of the run is also equal to 8. 8 by 8 will be equal to 1 and that is why you have 1 here. For a central composite design it can be shown that the moments are carried over up to order 4 and let us take the design matrix for the central composite design and use it to investigate the values of the different moments. So for the central composite design we can look at the values taken by different moments. All odd moments through order 4 that means orders 1, 2 and 3 are also included. It says that is why through order 4 that is the moments that contain at least one odd power like I or I cube or I squared j. So there is at least one odd power corresponding to the power of j and then I j k and this is all completely odd moments because I is different from j and j is different from k and I cube j and I squared j k are 0 for I is not equal to j not equal to k. So tables are shown in the next slide as examples. So in this design it may be easily visualized that only non-zero moments for k is equal to 3 or I squared, I squared, j squared or I cube to the power of 4 for all I not equal to j. So we will continue on this topic after taking a break.