 Hello everyone, today in the previous session we have discussed different components of time series analysis that means what are the components exist in a time series data and how can we analyze them and how can you make a prediction by understanding that behavior of the data that part we have discussed through different components of time series data. Now we are going to discuss the measures of forecast accuracy that means once you make a forecast with the data after understanding the different components and the pattern of the data and bringing a accurate model or appropriate model time series models, suppose you have made the forecast right and that forecast is not sufficient. So you have to calculate the measures, the accuracy level of your forecast right. So that we are going to study now, what is measure of accuracy? Suppose if you calculate mean of a data, suppose you have a past data or any data and you want to calculate the mean of the data, in that case you know only mean is not sufficient, you need to calculate the standard deviation right like variation of the data measure of this person you have to calculate. So this way you can calculate mean variance and standard deviation and that standard deviation or you know diversion from your mean data is also important and you can understand what is the confidence of the data or confidence interval of the data and accordingly mean and standard deviation people calculate right. Similarly in time series data, when you have past data and you understand the pattern of the data and you have already selected, suppose you studied all the models of time series and you have already selected a model and you have made a forecast, but this only forecast or expected prediction is not sufficient. At the same time like mean and standard deviation, at the same time you need to predict, you need to calculate the measure of accuracy, the error part how much deviation are being there with your data from your mean predictions. So that is called measure of forecast accuracy. For example, if you go to the time regression analysis or causal models which we will discuss later, if you calculate the regression or the relationship between dependent variable and independent variable, so we calculate how strong the relationship between dependent variable and independent variable right and that calculation are been called as a measure of determination or you can say r square. So r square which is nothing but square of correlation coefficient, so this r square actually measures the confidence or how much it is explaining, the independent variable is explaining the dependent variable, so that is been calculated through r square, so that define the relationship between dependent variable and independent variable. So where the r square means the better the regression analysis or the relationship you have built between independent variable and dependent variable, same logic need to be implemented into time series data also. That means if you have a past data and you want to make a forecast for future, you have to calculate the measure of accuracy, how the measure of accuracy can be calculated we are going to study now. So generally there are 4 methods, 3 methods, there are many more methods available in the literature, but today we will discuss only 3 plus 1 total 4 methods, 1 is the mean absolute deviation of the data and mean absolute percentage error, then mean square error and then square root of mean square error, we call it as root mean square error RMSC which is very popular in financial domain. So let us understand one by one the measure of accuracy of the data and how that is been calculated and let us bring a numerical example, through that we will understand how this measure of accuracy of a time series data can be calculated. So first one, the mean absolute deviation, look at the calculation here, it is nothing but the data, the actual data, the difference between actual data and the forecast data and take the average of all the error, the residual part and that average if you take that is called the mean absolute deviation. I will give you example in the next slide you will get to know, suppose you have a time series data, let me put a sample data and suppose this is your forecast, this is actual data and this is your forecast data. Suppose actual data are here, suppose y1, y2, y3, y4, y5, y6, dot, dot, dot, dot yn. So you have the actual data and suppose you have made a forecast, y1 forecast, y2 forecast, y3 forecast, y4 forecast, y5 forecast, so this way suppose you have made the forecast, right. So now if you calculate the error term here, in the right hand side column, I show you in the next slide or in the excel, so you calculated the error, right, error, what is that? What y minus y hat, so this is what your you know formula. So if you calculate the error term, this error term if you consider suppose error, this error term if you consider and if you calculate the MAD mean absolute deviation, mean absolute percentage error, this will give you the accuracy level of the model. If the total error, if you take you know some of them we call it a total error or the average of the error, but that may not be good to make a prediction if you take the average of the error, because what happens you know there may be you know this in this case suppose you have a plus 8 error, but suppose here suppose e10 suppose you have a say minus 8, so effectively what happens if you add them you will you will say it is cancelling out. So it is coming to be 0, so error is not 0, but actually too much of deviation from your actual data, right. Suppose you have a data like this and your forecast are like this, so look at the deviation, this deviation if you add your total error might be if you take only average it might be you know 0, so it is not good, but your prediction should be closer to the line. So therefore, we take the absolute value of the error and then we take the average we call it as a mean absolute deviation. Then come to the mean absolute percentage error, it is nothing but the absolute deviation part, but what we consider we take the absolute value of the error the residual part and then we divide it by the actual data that means relative value how much error is there in terms of percentage and then we multiply with the 100. Suppose I will show you in the next slide suppose here you have calculate the error error on. So error 1 by y1 into 100, so this is nothing but your percentage error of actual data. So error is the first period error say and this is your actual data, out of actual how much is the error is occurring the deviation has been there, so that you multiply with 100 you will get the mean absolute percentage error. So absolute percentage error you will get and after that if you take the average of the data you will get mean absolute percentage error. Then next is the very popular and throughout the forthcoming sessions I will focus mean square error or RMSE, but it does not mean that RMSE or mean square error is you have to follow you can follow any one of them because in industry people talk about how much is the error percentage error. So they call it is a 10 percent error 20 percent better, so this way mean absolute percentage error has also merit we can follow that also, but I will be focusing majorly on mean square error because in the calculations of excel I have preferred the calculation through mean square error or RMSE or minimizing maximizing etc. through optimization software. So think about mean square error, so it is also same like you know like the waste standard deviation in calculation you take the error take a square and sum them and divide by n you will get the mean square error square of the error. So if the projective negative part I mean remove now you have taken the square and then you are taking you know average of them sum of them and then you do the average you will get the mean square error. RMSE is more interesting because you take you know square root of MSE so that is called the RMSE both are not linear, but very popular in optimization process or you know in time series data analysis for making a forecast with supporting document that is called mean square error or RMSE as a measure of accuracy which provides the strength of the reliability of a forecasting models. Let us understand these three or four you know concepts of mean absolute deviation, mean absolute percentage error, MAPE and then mean square error MSE and the RMSE root of mean square error through a numerical examples. Here let us consider the basic model or name model because we have not started the you know different models of time series in the next session onwards we will focus on that, but let us understand the basic name model which does not require any calculation only. Name method says that whatever the past data you have past period just immediate past period that is your forecast look at here whatever has happened of say TCA stock price in the last day that will be the forecast for the next day. So that means today closing price of say gold or say you know crude oil price or say stock price so if that is the today's stock price closing price tomorrow will be the considered that price will be considered as a forecast for tomorrow no calculation no average no you know seasonality nothing you are considering just you are considering the past data as your forecast then the difference of them with the actual and the forecast will be your error. So these errors will be considering through name models and we will understand the four method of measure of accuracy. Let us go here you know even this name method is also very popular you might say it is very easy you know let us go to the next slides we will get to know you might say it is very easy say look at here so you have a say first period of first week say is 23 the next period forecast this is actual actual next period forecast is say 23 so this has come now. So error you can write down error so error would be say 24 minus 23 equals to 1 so next period you can see for the third period the forecast is second period actual data is your forecast. So your error will be 32 minus 24 so it will be say 8. So this error you are writing in the right hand side and you will calculate the average or mean absolute division etc. etc. RMSC etc. we will calculate through Excel. So this is what called name method for basic understanding of the models I have bought name model but it has a good merit also I will discuss in some other session the advantage of name model. Actually you know in industry the benchmark has been done through name model though it is says that you know whatever has happened in the previous period you consider that as a forecast no calculation are required. But if you go to the industry say quarter to quarter which is forecast or the sales as well as the earning or say you know EBITDA and the PAT profit after tax so all these things are being predicted or been measured through a name model actually. So that means what happened in the previous year corresponding quarter. So that data are being considered as a benchmark for you know improving the you know EPS or the PAT of a company. For example say suppose you think about Hindustan Unilever HUL. So suppose HUL forecast for say third quarter you want to make prediction right. So what whether it is good or bad how will do the prediction can be done some other way but benchmark analysis can be done or the comparative analysis can be done through name model. For example what happened in the previous years corresponding third quarter sales of HUL or say PAT of HUL. So that is your benchmark and that you are making a comparison with the current period of third quarter of this year 2023 say and you are making a comparison. So this is what happened in the last year corresponding quarter that you are bringing here and you are making a comparison you are seeing how much is the up how much is the how much whether it is a positive or negative. So this down or up that analysis can also be followed under name method. So previous quarter also you know for margin analysis you can do what happened in the second quarter of HUL and that data you can bring like just immediate data immediate period data and you can see what happened in this current quarter and you can see the difference also. So for margin purpose this is good but for you know profitability and growth purpose of a company people use name model people bring name model because there is no calculation only what happened in the past that you take and you make a forecast for the future. So these are you can make a comparative analysis with the past sales of a company. So this is called name model. We are not focusing on the name model we will bring this name model as a past whatever has happened in the past that concepts only and we will make a calculation of four method. Look at the data here. So we have the data time series data so sample data I have prepared. With this data we have used the name forecast only. So suppose whatever has happened in the past that we have taken as a forecast for the next period and we have calculate the error in this column. So you can take the average of the data. So that we will not consider I told you we will take the absolute value of the data. We will take the absolute value of the data. Absolute value of the data and then we will make the forecast for the we will calculate the MAD for that data that error data. So here is the absolute error and if you take the average of them you will get the MAD. Suppose 2.5 for this data and the next period forecast is the immediate past. That is not important for us to understand because this is just a forecast. So for any data we will be studying several dozens of say time series models or forecasting models. So everywhere we are not bothered about what is the forecast value. Our main concern will be what is the error part mean square error or RMSE or the percentage error because lower the error that is the better forecast because forecast is prediction future prediction. Even in the Masenland it takes all about the predictions. It might not happen in future. If some computer closes store enters demand might shift to you so your sales might go up. So therefore you cannot make anything everything accurate. So you just a forecast by understanding the study of past data or behavior of the past data through different understanding of components of the data and you make forecast. That's the forecast only. So to make a business plan for the forthcoming periods or quarters but effectively you need to focus about absolute error calculation. Suppose here MAD you have calculated. So note that point actually. Now if you take the percentage of error so you know percentage you have taken and divide by absolute error by actual and then you multiply with the percentage error. So you will get the absolute percentage error and then if you take the average of that you will get the mean absolute percentage error. And the square error you can take also square of the data or absolute data, absolute error square and then if you take the average you are getting RMSE, MSC. And the square root which MSC is RMSE 3.22. Suppose this four you have calculated. Now you might say, sir okay we calculated the MAD of the data with actual data and forecast data by calculating the error. We have calculated the mean absolute deviation. We have calculated the mean absolute percentage error. We have calculated the MSC and we have calculated the RMSE. But the point here is that which formula we should calculate we should consider for our forecast with our data, with industrial data or practical data or say for our project. You can select only one of them. You don't record all four. MAD is also has a merit percentage error. MSC also has a merit. MSC or RMSE also has a merit. Any one of them you can select. You don't record all three to calculate for a particular data. Because these are the different model because here is a percentage. It's a 9% say. And here is a 10 point something some different calculation. So you cannot compare them. If you what you can do is that you can select different model. Suppose name model for that data I have used and have calculated the error. Suppose if you calculate the say moving average model suppose if you bring a moving average model and you calculate this particular data forecast through moving average you will get a different percentage error or say MSC or RMSE. That RMSE of moving average model and name model you can compare. And then you can see for this particular data whether moving average model is better or name model is better. That is that comparison you can do. But among the measure of accuracy formulas you don't need to make any comparison because they have a different merits. So you select any one of them and you make different and use different models and make a comparison and see whoever is providing lower RMSE or the lower error percentage error you select that model okay. So select only one formula among the three or four that I have discussed today. So let's see suppose for this particular data we got name forecast with percentage error of 9.34 or say mean square error say 10.37. Note down it. Here you can see the name model also. For your understanding you can see whatever has happened in the past that is the forecast for the next period. Look at here whatever has happened in the past that is the forecast for next period. Whatever has happened in the past that is the forecast for next period. So this way you can culture and the actual is coming like this. So error you can write down which I have shown you in the slides. Error you can write down in the right hand side and you can calculate the name forecast. The forecast is not also bad. But it does not take the trend of the data whatever has happened in the past that you are taking. So seasonally trend cannot be captured through that effectively. It is just a basic model to make a comparative analysis of the data. So I thought of bringing that to focus on measure of accuracy calculations. Now for same data suppose you know if you use with moving average model suppose if you calculate the simple moving average model for the same data if you implement the simple moving average and if you calculate the error I will discuss detail of moving average models simple moving average and the say you know weighted moving average or you know exponential moving average in a separate session. But here you see suppose you have select the moving average model and suppose you have considered four period moving average. So, you take the four period average and make that as a forecast for the next period. Then in the next period what you do? You drop the older period and select the this four period average you keep the range you keep and then as it is then you can freeze it and then you consider the next period into your data. So, your next combination will be like this to make a forecast for the sixth period. So, this way suppose for example, for 14th period your forecast will be combination on the past four period because four period combination you have taken. So, you can make a forecast for that and if you drag that you will get the forecast the moving average once you select the cluster or the range of the data say four period and if you drag it every time one older period you will drop and new period you will be considering you will be adding to your average. So, this way if you consider and if you make a forecast that forecast is called the moving average forecast. We will discuss that detail in the next session. Now, using that model for the same data if you calculate the absolute error error and the corresponding absolute error you are getting MAD is 1.71 and percentage error is you know 6.61 and RMSE, MSE is a say 4 and RMSE 2. So, what is the comparison now? Now you can compare think about you know say MAP percentage error think about this how much is the percentage error for moving average data for moving average model of the same data 6 percent 6.6 percent if you select the percentage error as a comparison. Now if you come back to the name model what was the forecast through percentage error? The percentage error was 9.34 and if you see name model sorry moving average model here 6.61. So, which model is better for that moving average model is better because it is giving the less percentage error 6.66 percent 6.6 percent and that was 9 percent 9.5 percent say. So, the name model is providing higher error. So, therefore, do not select name model for this data you select moving average model for this data. This is what the comparison you can do for that particular you know for a particular data and by selecting different models, but you cannot compare MAP and MSE. For example, here you can see the MSE is 4 here for this moving average model of this data 4 note down it, but if you go back to name model here you see for name model it is 10 here it is 10 and there it is 4 only. So, which model to select the error is where you are getting less error on an average you are getting the less error through moving average model for that data. So, do not select name model for this particular data you select moving average model. So, this way you can use the measure of accuracy for different model. So, today what we have discussed actually. So, here you can see the graph of moving average model and you can see it is taking the average. So, 4 pure average it is taking and it is forecast the forecast started from that point and then you drag the forecast the moving average you get the forecast. So, this question looks better actually as compared to name model and this is the forecast which I have shown you through competitive analysis here. So, this is called the different aspects of measure of accuracy whether it is a mean absolute deviation, whether it is a mean absolute percentage error, whether it is a mean square error or RMSE. I have not talked much about RMSE today, but next class onwards you will see everywhere I will be using RMSE only for my excel calculations. And then you can see the competitive analysis between different models through percentage, so through error of absolute error or percentage error or say mean square error and that will be your benchmark calculation for a model which model is most suitable for this particular data. I will show you now some excel calculation for that error calculation here you can see. So, here I have calculated the name model enter same data what I have shown you in the PPT I have kept in the excel now this is the data you can see and this is the forecast for the next period just I have copied this 23 here and then 24 here this way I have copied here you can write like this also like this cell is your forecast and you can drag it right you can drag it. So, this is your forecast right, but look at the error. So, error I have calculated here and I have taken the absolute error here absolute error here right absolute data of that this absolute value you can keep here and then percentage error I have also calculated here MAD mean absolute deviation of the average of the error and here I have calculated MAP in terms of percentage and the average of them and here I have calculated the MAC what is that just you take the square of them square of the absolute data or error or actual error because you are taking the square and then you take the average of them total 16 error I guess total 16 error we have a 16 error. So, this error is the main document of your residuals of the data and using them you can calculate the MAD MAP and MAC and the corresponding you know RMS also you can calculate square root of it. So, all four are been calculated here this way you can calculate the error for any data with corresponding forecast. So, what is the summary this is the forecast 25 for the pre-auditing and this is the MAC 10 or say MAP 9.34 percent. Now, for the same data I have used the moving average and you can see the forecast is 26.5, but in main method here it is 25. So, moving average model I have used so 4 pre-auditing average I have taken 4 pre-auditing average I have taken and I have dragged it right dragged the moving average. So, I got the forecast for the pre-auditing. So, here you can see 26.5 is not the main part it is just a forecast for throw moving average model and in name model what is the forecast in name model the forecast is say you know the forecast for name model for pre-auditing are just you know 25 that is it. So, you might say here it is 25 there is 26.5 which one is better both are better both are good that is not the objective, objective is the calculate the error. So, here you can see for moving average the error is 6 like here you can see it is about 6 percent only right almost 6.6 percent here and here if you see the name model what I talked about it is a 9 percent. So, moving average is better similarly if you think about name model MAC it is 10.37 or RMS is 3.2, but if you go to moving average you are calculating the MAC or you are getting the MAC how much 4. So, 10 and 4 it is a much difference right. So, it is a much more this model is much more reliable or with better prediction you are coming up with moving average model for this particular data. So, you can recommend that for this data select the moving average model because it is giving lower error or better accuracy. So, this way you can actually you know calculate the or use the measure of accuracy for different model and this is a mandatory for any time series data without that you cannot make prediction only the outcome of the data for the future. That means 26.5 is the forecast for the pre-audiative that is not the sufficient at the same time you have to calculate the measure of accuracy whichever the model will give you the lower a measure of accuracy that model you have to select. Suppose for this particular data we have is name model and we have is moving average model and we found moving average model to be the unique winner. So, you select that, but if you select any other model which can give a better forecast than moving average model you have to go to that model rather than moving simple moving average model. So, in the next class we will discuss different type of moving average model and with the same data we can bring and we can see among the moving average which models which model is more suitable for this particular data. We will understand more detail about that in the next session, but today let us conclude the session about understanding of different type of components of time series data which we have discussed in the part 1. And now in part 2 we are completing the you know the different type of measure of accuracy and we understood how that this measure of accuracy are being calculated and what is the need for their use in time series data. So, with that let us conclude today's session.