 Then, why exactly the kind of data which we have is unique in some sense and what difficulty it causes and then the kind of tricks or the theory which we use for getting better results and then some little bit of work on some more cutting edge and newer techniques like multiple learning and dig learning. So, essentially the idea is of machine learning is you have a set of, essentially you have a set of vectors, x is a matrix like each row is a vector and then those vectors get mapped to a function. So, essentially what you want to identify is what that function f is from the multiple points which are given to you from the data set. So, if fx is discrete 1, minus 1, 1, 2, 3, 4, then it's a classification problem between your specification problem and so on. So, for financial stuff, the problem is as simple as I am trying to predict how many of you know what nifty is. So, basically it's an index which is traded in Indian market. So, the problem could be as simple as I want to predict what's going to happen to nifty index from today to tomorrow. So, it's as simple as that. So, whether it is going to go up or whether it is going to go down and you know I mean like this problem if it can be solved with a reasonable success can make a lot of money obviously because if the market is going to go up, tomorrow you can buy it today and then sell it tomorrow at a higher price. It's going to go down then you can sell first and then buy tomorrow. So, it can make a lot of money also. So, essentially you know the thing which we will try to see is related instrument, financial instrument like nifty will go up or down in the next period with what probability that it will go up or down. Is it possible to design algorithms which will be more accurate on larger moves? Obviously you know if the move is larger then you have an opportunity to make more money on that. So, you know I mean can you design an algorithm which actually goes wrong but goes wrong on a smaller move and you know when the move is large it goes more right. How to select X for a given security? So, you have to predict nifty. How do I select the attributes? You know I mean what are the attributes that are important? For example, you know somebody was talking in the morning about like if you want to predict something about the user of a website then you see which website that user has previously visited and on this particular website what that user has searched and so on. So, these are like in machine learning terminology we call it attributes. So, how to select these attributes so that it becomes easy for predicting the you know binary move whether it is going to go up or down. So, that becomes you know so that is where all the domain knowledge this is selecting X is where all the domain knowledge comes into picture. So, everything else is sort of done automatically by the machine learning algorithm. So, your X is a parameter that you use right? Yeah. So, there is lot of thing that happens in financial instrument happens in real time. Yeah. Like if the finance minister says something. Yeah. That is an X. So, depending on what kind of problems you are focusing on that could be an input we are not using that we are not using text mining kind of a word. So, for example you know I mean what financial finance minister says or you know what company CEO says when the things come out. You know there are companies in the world not in India I think but like there are companies in the world who actually do text mining to analyze you know whether the see whatever CEO is saying whether it is a positive thing for future stock price or negative thing for future stock price and you know those can also be used as attributes over there. We are not doing that right now but you know that can be done. Another thing is which we are learning we should choose. You know I mean like here again as I said you know it becomes very important because any small improvement we can get in the predictive accuracy goes a very very long way in terms of making what we call a strategy of prediction from loss making to profit making and if it is already profit making then it is like profit in business and then you know that can be a brilliant brilliant and call the billions of dollars at me. So and like you know for learning algorithm how do we at least check whether it would have worked historically. So essentially you know I mean like financial markets have a history of let's say five or ten years like you know the relevant history or what would you would like to do is you know you would like to design an algorithm and then at least test whether it would have worked historically we had applied it at that particular time and you know I mean like if it would have worked historically then there is some chance not directly and it will work in future also but if it if it didn't even work historically then there is no chance that it will sort of work in future. So I just wanted to show this graph in terms of you know what we are trying to predict. So essentially you know so see these are the daily movements of nifty so nifty index and you can see how what I mean. Essentially what we are trying to each of this thing is actually a day and on that day we are trying to predict whether it is going to work or whether it is going to go down. So it just keeps like you know it looks like a random noise so you know I mean and that is what it makes it's so difficult to you know predict like what's going to happen. Also usually the financial instrument data, credit financial instrument data is what we call e-condition. What do I mean by e-condition? If you actually you know take all these returns. So return is simple basically tomorrow's price or today's price minus yesterday's price divided by yesterday's price that is a percentage difference. So if you take an average of that it is like you know over this 4 period it is 6.7 e raised to minus 4 but if you take a standard deviation of this data then it is 0.016. So if you take a ratio of standard deviation to mean that is 23. So because of this what happens is there are every day the index is moving a large amount but you know over a period of time because on a given day you may move positives or negative over a period of time actually most of the movements cancel out and then you know you get only like you know 12 or 13% for annum kind of return like over a long period of time. So for example you know I mean this is 1.6% so a daily move of nifty can be about 1.2% and you can imagine there are 250 trading days in a year so if you could capture every day's moves correctly then you would be making 1.3 multiplied by 250% return in one year. So nobody makes that kind of return right? I mean that is like 4-500% return only on one single instrument and because nobody can get it accurate 100% nobody can make that kind of return. And nobody can get accuracy to that level because of this move. Since the average is the signal is very very small and the noise is very very large. In 2006 you know what that means it is basically the fourth movement of the probability distribution that is very large. If you have a Gaussian distribution which all of us know this process should be 3 and this is where so that means there are many many very large moves. Also if you have about 6 sigma events and stuff like that then 6 sigma event is basically 3 and then 3 times more likely to have a 6 sigma event as compared to you know Gaussian distribution. So all these things are included in non-stationarity. Basically if you see there are small moves over here there are large moves over here because of that the distribution from which these moves are coming from also changes over a period of time. Any questions? So x axis is just space. So basically this is January 2003 I think. So that is January 1st January 2003 and this is like you know three days back when I actually created this presentation. Another thing is you know I mean like one of so this is what we call is the time series right. So one of the ways of analyzing whether there is a predictability in time series is what we call auto correlation. So if you actually do auto correlation of these returns then you will see that there is no auto correlation. So basically what happened yesterday is of hardly any you know I mean like information to decide what is going to happen today. And this is why you know most of the linear methods will become almost useless as far as you know I mean like especially if you do it as a univariate analysis. So if you just use nifty data of last 5-6 days and try to predict what's going to happen tomorrow that is not going to work very well because you know there is no auto correlation. So there is no dependency on what's going to happen tomorrow over last 1, 2, 3, 4, 5, 6 or whatever 20 days. Another thing is what happens is suppose you know by whatever method I want to do. If I am able to build a prediction system which does about 52% accuracy of prediction then because of the transaction cost etc. that actually system either loses money basically I have 100 moves on those I need to be right only in 52 times and then around 48 times so essentially I have a net gain of 4 moves right. So 4 moves multiplied by 1.1% on an average per day which means 4.5%. So 4.5% over 100 days is approximately the transaction cost you made up giving in reading this particular instrument and that essentially is breaking even the losing slight amount of money. But suppose you could build a system which is only 56% accuracy which means only 4% more accurate time you know the break-in system then this actually makes a phenomenal amount of profit. Why a phenomenal amount of profit? 56 minus 44 is 12. 12 multiplied by 1.1 is about 13. So basically 13% return for every 100 days. So in a year there are 450 days so basically in a year you would be able to make approximately 30% return on analyzing. So which is I mean anybody would love to have investment which actually generates 30% return year on year right. So the reason I am showing you these numbers is because you know it is we are playing so close to randomness essentially our system completely you know random would be 50% accurate and system which is you know slightly more predictable is 56% accurate and a system which is as close to randomness is able to make money. So it is very hard to actually differentiate multiple systems which are random from each other which is actually a real prediction and which actually is a random system and that is my next slide. So here what I did is I took that data of 50 and I randomly generated plus 1 and minus 1 for those 2,000 plus points 100 times. So basically I had these 100s quote unquote systems which were random systems and I chose the one which had the best accuracy out of that. So these are like you know there is more intelligence over here I am just creating a random signal of that thing and choosing the one which gives me the most accuracy so that is the blue line. So you can see you know it looks as if you are going to make money using that particular system but it is a randomly generated signal so you know it has made money in sample but obviously it is not going to work if you start using it out of sample. So that essentially is a big problem right because you could do so called machine learning and you could do all kinds of trading and technical indicators and this and that and you could maybe do it many many times you have a certain amount of data and you say that okay I mean if somebody any of you guys trade then you know you use moving average or something like that and you know you change 6 weeks moving average 4 weeks moving average you change it around and see which one gives you the better accuracy. So you can think of this if there was no predictability in financial markets and if you were doing this many many times so let's say you generate these 100 random systems just by this logic you are actually going to get one case which is going to give you a profitable system in sample but when you put it into the market it's not actually going to make money for you. Any questions on this? So okay so if this was much higher so basically this is the first line so let's say if it was at 0.4 that means what's going to happen tomorrow is positively automatically what happened yesterday so if yesterday's return was positive then it's going to happen that tomorrow's return is also going to be positive if it was below then it means it's going to be negative why is the strength by which you know the adjacent moves are correlated with each particular one? same about the 100 moves every time you are making the same amount of money you are making the 100 times sorry you are saying 100 means right? 100 means you are saying the same 6% is accurate so you are saying 100 times? no so what I am saying is if you have 100 moves the average size of each move is 1.1% average so some of them are much smaller but the average size is 1.1% so on an average if you were going wrong uniformly independent of the size of the move then what would happen is 56% accuracy system would actually end up giving you what we call edge of 56-44 which is 12% so basically for every 100 moves we are doing 12 moves better than you know I mean like basically there are next 12 moves which we are going to make money basically 44 moves are going to be positive by random chance 44 moves are going to be negative by random chance and 12 moves essentially we are going to be able to predict and so this 12 multiplied by 1.1 becomes approximately 13% so for 100 moves which is 100 days which is approximately half a year in half a year you actually are able to make 13% so just yeah in 100 moves you were saying about because I am like saying investing in one leg for 2 minutes so 100 moves should be equal right and you are calculating that to a 13% so every day I need to invest one leg so then I get yeah I mean there are different ways of investing actually we are not talking about investing right now but what you could do is you could invest 1 lakh rupees or 1 rupee every day and then in 100 days what you could end up doing is if you are investing 1 rupee then you would make 13% so that is all so the classification from the only classification is the direction of the 100 but doesn't classify the magnitude so if you have the top moves which are smaller in magnitude and the top moves are relatively larger in magnitude then you would have different results that is not I mean like see basically we are not actually talking about whether the move is up or move is down whether we are more accurate on what kind of moves because we can make money from up moves also and we can make money on down moves also so essentially you know as long as your accuracy is same for both kinds of moves you would on an average rate of making the same let me give this question which is the most if you consider the cases 36% you are accurately able to predict you know the direction which is what you are predicting not the magnitude not the magnitude because direction prediction is hard enough so magnitude prediction becomes even more difficult but there can be a regression type of models where you know you are trying to predict magnitude and then do investment based on that also but right now we are mainly focusing on direction prediction but we try to take care of magnitude in some other way so efficient markets are mostly like that mostly in academia nobody believes that otherwise this whole financial industry of real world dollars would not exist so that is mostly for academic reasons and that has been put there because you know I mean it makes you know it makes you know it makes you know it makes you know it makes you know it makes you know it makes because you know I mean it makes proving certain theorems easy and actually efficient markets mostly are used in a way that efficient market is an equilibrium state so that actually gives you a way of measuring whether the current condition of the market is actually much away from the equilibrium so you know it actually works as actually a trade generation mechanism because you know you are not you are not in the equilibrium state so you will be able to actually you know measure how far you are from the equilibrium state and then take a trade on it so it is probably a little complicated but you can talk offline later the absence of auto correlation does it imply that you cannot keep the stock distribution much longer? absence of auto correlation implies that we cannot use linear univariate methods to try and predict this particular security linear univariate so basically I mean you know I mean I could have a variable which is not dependent on itself at all but it is dependent on something else which I have access to so that is a multilateral problem but I can't use the same graph so I can't use you know last 4 days of nifty data and expect that and I will be able to predict I am sorry I can't use this for univariate univariate means if I am looking at only behavior of nifty then I don't have a chance of predicting nifty in theory by using linear methods nonlinear method there could be I mean because auto correlation only works for linear modeling methodologies so it's just a normal pseudo random number generator like you are trying to predict how it is so that is the nifty data is not normal I am only see basically I am randomly generating whether it is plus one or minus one so whether I want to go long today whether I want to buy the security today or whether I want to sell the security today so the random number generator normal random number generation which I am using actually has more relevance in this particular case because I am only doing the signals I am taking the sign of the random number that actually so that's what I am saying my simulation process and my historical data are not really connected so I mean we can maybe talk about this often later yeah so that I mean like that probably a little bit ahead you will get to see what kind of stuff which we are trying to do that doesn't mean that that's the only thing one can do okay so exactly to answer your so as I said you know I mean like the auto correlation essentially shows that you know you can't use historical data of 50 returns to try and predict what's going to happen tomorrow but it may be possible that you know I can predict what's going to happen to Nifty by using what's happening right now in US markets or in Asian markets right now or in European markets right now so one can actually use all these data together and then create that attribute matrix X and then try and see if there is any predictability in that so for example you know I mean like theoretically not theoretically but practically speaking if any of you are trading then you know that what happens in US market yesterday becomes very very important to what's going to happen to Indian markets today so just I mean like one things that that should be the case so you can use that particular fact in your data that you say okay I am going to use what happened in US markets yesterday as one of my attributes to try and predict what's going to happen to Nifty from today to tomorrow similarly you can do it let's say you know what is happening to gold is very important to you know what's going to happen to security markets because you know usually they happen opposite relationship whenever there is a danger that economy is going to fall into inflation or something like that gold is going to go up and then you know equity markets are going to go down there is an opposite relationship so let's use the gold what's happening to gold right now and then use you know some kind of machine building algorithm to actually understand what you know how that effect actually gets modulated another thing is you know as compared to you know most other problems one difference that we have is we have a time series data so essentially you know it's not like you know one instance of a user is completely independent of another instance of another user actually you know there is a time chronology which happens basically you know you highly should not be using you know what behavior market has in 2005 to analyze what behavior market would have had in 2003 because you know trying to predict what's happening in 2003 you know you did not have access to what is happening in 2005 so you know you should not be using that so that is what we end up doing it's called walk forward what we have done over here is we have arranged the data so this is each of this line is the row of X and this plus or minus is whether on that particular day market went up or down so that is 2003 and this is yesterday essentially so what we do is this so it takes a back testing process so we take some amount of data say 2 years of data you know then use that data build some models so let's say you know something simple like logistic regression or support vector machine or whatever you want to do build that model using that data then predict for today so no historically maybe you were right using your model or you were wrong you just keep track of that and then move your window forward by one point repeat the process and then what these signs are these are what actual movements where is what characterizes your back testing program so historically if we had applied this strategy then how it would have worked and whether it would have made money for us in the market or not and if you see each of these prediction is completely out of sample so you know so there is some hope that whatever we are doing over here actually has some hope of working in future so this is probably going to give to sleep I will not talk about it too much but you know I mean just to just as a preamble to my next slide essentially you know I mean like in machine learning this bias where I am straight up actually is something very very important I mean most of us actually don't pay attention to that because you know usually from prediction perspective most of the real world problems are fairly simple so whether you use linear models on that or you know whether you use nonlinear models on that doesn't really matter but since we are working here very close to randomness it becomes very important so what happens if you use something some very simple model like a linear regression model and then whatever comes out of that you take a sign of that so if you use that kind of a model then what we call is that model is extremely biased so there is no hope that whatever is the relationship among the variables which we are using to predict that actually will be sharply captured by a linear model because we know that the world is not linear now if you actually make the model extremely nonlinear then what happens is like there is a hope that the actual function which you are trying to approximate is part of this whole nonlinear class of the models which we are using but what happens is this set becomes very big and then it becomes very difficult to actually pinpoint which model you are trying to so this is a bias and this is a variance so because you have given a certain amount of data and we are trying to identify using that data what that model parameters are essentially this set which is the variance this becomes much much bigger as compared to that and the theory goes that bias plus variance is more or less difficult to manage it is difficult to come up with algorithms which reduce both bias as well as variance so what we try to do some of the things which we try to do to try and say let me try and emphasize over here the motivation is that if we can do something which robustly improves our prediction accuracy by even 1% or 1.5% that is actually a very big deal for us because it ends up increasing our returns by 4% or 5% per year and increasing annualized returns by 4% or 5% per year is actually can be a very large amount of money so what we do is I mean bagging probably you guys have heard about bagging so what bagging is essentially you have the data set you essentially sample some data sets from that use the same methodology and then whatever are the samples you essentially do the averaging of that and then use that as a prediction then multiple learners on those with different underlying theories so let's say you have a support vector machine larger simulation has a simple similar philosophy but then you have random forest or trees or you have a basin approach of things so you have been given a data you apply these various models to it you try and create the same thing out of that because in this scope also randomness sometimes SVM will be correct sometimes logistic regression will be correct sometimes your random forest will be great correct and then when you combine these there is a hope that you know I mean this combined consistency which is an averaging system works actually better in practice then multiple kernel learning I talk I don't know if I'm going to get time but I talk little bit more detail about that then combining unsupervised and supervised learning so essentially your data set may have some clusters so you can actually train your supervised learning model on one of the clusters and you might actually do better on that boosting if you know what it is it is not very useful because financial data is very noisy data and boosting generally doesn't work with noisy data sets in general you know whatever you find in theory generally you will see that things work a little bit worse on financial data because of the amount of noise that the data has also another interesting thing is market learns and trades against you so basically if you are running a successful algorithm for long enough time other players in the market so essentially financial market is trading for you to make money somebody has to boost the money obviously nobody likes to boost the money so they really permanently try and you know learn what you are trying to what you are doing which makes you make money like you know in the long term and then they learn that and they trade against you so basically if they know that you are going to take a long position they may actually before you actually take a long position and take the price of the security up before you actually buy no I have not used it yet but there are people who use it so there are the let's481s components of people use that too so essentially take a line from one of the movies three things are most important averaging, averaging, averaging so actually three things are like really bad also optimization, optimization, optimization so if you actually try and optimize your model a lot it actually works really badly for financial markets in out of central averaging is a really nice process too so as I said combining multiple methods generally you know so I am being very secretive not very but this is secretive over here I am not telling you these are the real research I am not telling you what assets I have applied it on and I am not telling you what methods I am using but still the point I am making over here is you know this is the kind of improvement we look for so like if you see this girl this is the averaging process, the black one and you know I really like this because across the methods it's actually outperforming the individual methods which are part of the averaging process so you know if actually this method performs very well on A6 but it performs badly on A1 generally I don't want to use it for A6 and discard it for A1 because you know I mean these are similar prediction processes and generally I want things which work across the board and this is my way of taking care of the fact that I am not over feeding the data and you know something is going to give me great results in sample but not so great results in out of sample I am sorry it is not in one case but in most of the cases where it is not better I think over here this one is good that is why so whatever Excel I mean I didn't change the numbers so now what I know quote unquote cutting edge method so these we really love right I mean basically because this is the cutting edge machine learning research and our data sets are not very large it's just that we have fairly small data sets like basically you know we may have 20-25 attributes and 2000-5000 you know sample points and from that we want to apply the best possible method which is out there which in a robust way actually guarantees us that our predictions are made so from that perspective you know we are doing what I would say a lot of core machine learning algorithms because our attention is not so much on you know I mean like how to clean the data and you know how to make sure you know some of the other things are like taken care of also another thing I should say we don't really do any fundamental research in our company essentially by cutting edge means you know whatever is just recently published in January 2014 or something like that that we try to implement in our work so I will rush through this because probably if you have heard about support vector machines you probably know about this so essentially you know it's basically a linear separator and then you know you orient the separator in such a way that the margin between the two sets is maximized that is the support vector machine logic it works very well in practice that was only linear if you want to actually solve some non-linear problem then you use coordinate for that so if you have this kind of a data set by the data set to classification generally you can actually transfer your data to some other domain and over there it's actually a linear separator this talks about the more mathematical part of the coordinate range so this kernel function which actually you know uses this is our original data set and then we are using some kind of a non-linear function to transform it into a non-linear space so that it becomes easier to separate the data in that space so there is a lot of theory behind this so essentially what it says is you know there is a theorem called Mercer's theorem which says that every machine semi-positive designate symmetric function is a kernel so essentially you know if you have this matrix if it's a positive symmetric matrix then it actually works as a kernel function on any data set now people use this fact and you know so this is actually the support vector machine so this is where the kernel function comes into picture and I am going through all this because you know I am going to motivate multiple kernel learning which we are trying to do so how to decide what kernel matrix to use there has been a lot of research and since the theory only specifies that K should be positive semi-definite people try to do something called semi-definite learning on semi-definite optimization on this you actually know essentially what you are doing you are trying to maximize you are trying to find the non-linear transformation mapping in such a way that the margin also gets maximized because of that non-linear mapping in addition to the margin of the data set so as you can imagine this is a very big flexibility that is there so what ended up happening is this problem actually overfitted so people could not get good out of sample results out of that and then recently there is something called ASO theory where you try and find a solution which is constrained in some way essentially one of the solution actually is equal to one and then people try to take use from that theory and then try to do the selection of kernel matrix and that is where multiple kernel learning came into picture so going back to our problem essentially what we want to do is let's say we have Nifty we want to do the Nifty prediction let's say we have a self-information attitude let's say last 4-5 days returns volume super interest whatever then there are supporting internet market variables maybe the inflation in the internet market for an exchange rate, USJ and R whatever that ladies now supporting all internet market variables so let's say US market, European market, Asian market so these x1, x2, x3 essentially you can define kernel 1, kernel 2, kernel 3 and then do x1 multiplied by kernel 1 plus x2 multiplied by kernel 1 plus x3 multiplied by kernel 1 and then work under this one arm of x which is x1, x2, x3 is equal to one and then you try and solve the joint optimization problem using that so that's the multiple kernel learning that's the fairly recent work which is out there in which we try to use it again same assets one kernel learning usually we end up using RDF kernel you know what that is and then multiple kernel learning we try to do on that and generally again the goal is to you know on everything you should do slightly later we have not been successful at it yet these are like bit lower than you know the one kernel learning but some improvements are there so we are not going to apply it into practice here because you know it's not consistently working as it is then if you have heard about something about deep learning that's actually a big deal in this race I don't understand that topic very well or essentially from a layman's perspective all I see that normal neural networks have one hidden layer deep learning networks have multiple hidden layers in between and what ends up happening is you know essentially each of these nodes starts becoming specialized for a particular type of sample so essentially Google actually did a big project like a couple of years back where they using this deep learning they identified which images had cats in it so that was actually something which we tried to do a little bit not at all successful at it but it's a race if you actually learn about it start a company you won't have a problem getting it funded somebody will probably in couple of years die for hundreds of million dollars also so somebody from you should try it and the good thing about this is that you know I will like what ends up happening is if you feed enough data to this then each of these nodes become specialized for a particular type of object so for financial data it could be like if you feed enough financial data then this node will become active for a certain kind of behavior of financial data this node will become active for certain other kinds of behavior of financial data and you know because of this I don't actually manually need to decide which attributes to use I can throw a lot of data to this thing and you know I mean hope that you know automatically it's going to understand what kind of combination of different data sets actually make things easier and then use that as a example yeah so people are talking about those correspondence I'm not sure how sort of relevant they are but you know I think Siri uses they claim to use some deep learning in recognizing the voice and stuff like that so so this is actually something you know it would be really good because this has become a real recently because it's almost uniformly breaking all the online competition records of accurate traditions of different problems so usually those records are held by support vector machines mostly and deep learning is actually going a few percentage points ahead of support vector machines and that actually is a very, very big thing so I don't understand about it enough so what happens is like if you put a lot of layers in the neural network it becomes extremely difficult to learn because you know it works on something called gradient descent and those gradients vanish when you actually start computing them because you have multiple layers those start vanishing and because of that it becomes computationally horrendous to keep track of you know all the calculations and then this Hinton is sort of considered as the father of this deep learning kind of a thing so he came up with a couple of you know new innovations which actually made that problem easier then that from that point onwards actually it started becoming a viable solution for different problems the evolution on the same I mean I don't know if there is like a fundamental difference between a simple neural network and a deep network other than you know the kind of algorithms you use to train them and make sure they don't overfit or you know they do the training so little bit about you know I mean I already talked obviously a little bit about you know what we actually do so we have multiple strategies which are running you know in the market and you know we are making money sometimes using money sometimes net net we are making a little bit more than we are using because we are in the market for last 4 years so I guess some money has been made and this is the kind of software we are able to do so this is the real graph actually this is the real money made this is the real behavior and this is how our portfolio has performed so you know I mean so this is our real there are no simulations and randomly generated signals and stuff like that that's it basically we have a small company of like 8 people and that is what we do day in and day out I mean just try and apply machine learning to financial prediction problems yeah sorry I didn't know what it is right? there are actually some of those A1 to A6 where come on what it is as you say this happening to other companies as well like you and you need them to see the performance their price will be 8000 if you can have historical data which is reliable and if you could find indicators so for example you know if you could find indicators of like you know how bad the whole prediction is in India so but if you had a reasonable and small prediction there actually it would be very nice to use that to actually do prediction of this now these predictions how you would use these are different story because you know vegetable is a large perishable so you couldn't actually buy it and hold it from experience other than the fact that you know when you're also reading you know non-label of things but that's another story so how you could use these predictions is a different story but the theory doesn't actually you know I mean like you could apply it if you I mean if you know that field well enough and is everything directed to make a difference? no I mean depends on so normally I would say you know I mean now in Nifty we are using about 20 different variables to try and create what's going to happen to Nifty today so you know so that will have some Indian market variables some you know inflation like variables some you know gold prices you know maybe copper prices maybe some global market whatever you can think of which actually you know I mean like so that process right now is manual so that is where domain knowledge comes into picture where you know actually I sit there and think you know I mean like what kind of variables would be relevant for predicting Nifty and you know then try out a few or there also I can't try I mean I can't say that okay these are like 2000 variables I will use 20 at a time and then you know I will put it through a simulation because that will become exactly same as you know doing 100 random simulations and choosing the one which is actually the best out of them so that is what I said you know optimization in this field actually is the worst possible thing you can you can't really do this where you know you are running 100 simulations and choosing the one which actually gives the best results and say that okay this is the one what I am going to include so how long do you think there are a day because like you said that an algorithm in a stock market situation has just like because there will be some others who start trading against that algorithm so how long do you think you like just saw that resistance so what is your strategy for so in theory machine learning should take care of that right because you know machine learning is what learns from the marketing so essentially somebody is trading against you but you are learning the fact that somebody is trading against you so you can incorporate that information in your machine learning your algorithm so for example let me give you a concrete example so let's say your signal is saying that if you buy something today and somebody already identified that you have a system actually which tends to say buy on such days so he comes to market one hour before and buys a lot of that quantity so by the time you buy it the price has already moved up but then if that happens enough times you will understand that somebody is actually doing that to you so now from here you should actually go short so you should sell that security because the price which has moved up they can apply this particular guy or multiple guys from there actually buy it tomorrow actually the price goes down so eventually that can happen so this process will keep on going until you know the market comes to complete equilibrium there is no under price or over price security but that something doesn't happen because there are so many different players with so many different philosophies and so many different horizons of you know there will be great people out there I mean I think still majority of them are human beings so that is where one tends to make money I mean it's not possible to make money and training against the machines here given that I am explaining there are no specific laws inside that rating there are no laws there is a lot of so there is a lot of the price of that how much they love you so it's very hard to actually do a general which actually will do better than inside of this but generally what we have observed is when we try to apply this kind of algorithm to use your stocks it was very difficult for us to make money because what happens if there are public investors who actually know the CEO of that company and they get the information so they move the stocks so it's very hard to put money on that but whenever we do this widespread security or something like that there it works because they are generally people who actually know the inside information but in reality it's very hard to do a general so that's why I am using that so so so so so so so so so so So this is actually up to now for us it has been mainly based on human intelligence deciding which attributes to use but that also process can be to an extent you know I mean like one can use statistical correlations and mathematical correlations for that or instead of auto correlations or you know I mean cross correlations one could use something like usual information which is more robust to know the effects of PCA PCA is something where you know you have a lot of variables and only some variables of course you think are important so you know so you do PCA and then you know it's a 20 dimensional system 20 dimensional PCA and then you see that only 10 of them actually explain the most of the variance and last 10 actually don't make any difference at all so and then you reject those Generally what we have seen is PCA doesn't work too well because the low information is what actually ends up making the statistical predictor So generally we have seen in our work at least that PCA has not worked too well Okay, thank you