 Welcome everyone to the session today. We have snippets from an algorithmic trading system in Kotlin. Denanje Nen is going to be presenting. We're pleased to have you with us today. Thank you very much. Okay. So welcome everybody. Today, we're going to be talking about, I have been working on algorithmic trading system. So we're going to, and that's written in Kotlin. So we're going to be taking a look at that and, not that entire trading system, but I've extracted some snippets out of that and we're going to take a look at that and take it from there. So I hope you find this useful session. The talk sequence is going to be like this. I think the first two or three minutes, I'm going to just spend on the financial markets and derivatives on options, just to give people a little bit of idea about that and why algorithmic trading. The talk itself is in three areas. One is I chose to just focus on three areas as opposed to the entire system, type safety using generics, then we'll look at a complex data structure, an auction chain trail, and then we'll be looking at concurrent programming using Kotlin's channels and actors. I roughly expect each part to last about 15 minutes. So I'll spend about 12 minutes and then from that point onwards, maybe take a break after each part for Q and A and then we'll move on to the next one. Now, the total topic is a little content heavy even though I kind of distilled just the essential code. So that essential code, I have pasted it and I have the link there. It's bit.ly at fnconf-algo-trading. So if you want to open it up on the side, feel free to do so. If you want to look at it later, feel free to do so because I'll be highlighting the important points, but if you want to kind of get into the details of it later on, that's probably a place to go look at, yeah? Okay. So first thing is about financial markets. I think most of us have some sense of, I mean, you can buy a share at 100 rupees or $100, maybe someday later, you hope to sell it at 110, but there are more advanced sort of instruments that get used called futures and options. And just an important, I mean, what an option is, I don't buy the underlying script of, let's say, Netflix. I don't even know what the current price of Netflix in the US markets, but let's just say it's $100 or we can use the Indian one, let's say 100 rupees. And instead of investing 100 rupees, you say, I just want to buy an option. I am betting and it's a gamble. I mean, at the end of the day, it's a gamble, though with a very weighted risk reward, saying that 100 is going to go to 110. So I want to buy an option, saying, I will Netflix, I'll buy 105, for what is called a strike price of 105. And if it goes above, I'll pay five rupees for buying that option. And if it goes above 105, whatever additional money I get in excess of 105, I'll keep it. And if it closes below 105, I'll lose the initial money of five rupees I paid as the option strike price. So it's a bit of a, but what it gives you at the end of the day is a lot of leverage. So I might go into an option, which is expiring, let's say just four days later, I might buy it for, so in this case, I bought it for five rupees. And that script moved from 100 to, let's say, 115. What I get at the end of the day is the difference between 115 and 105, which is 10 rupees. So on an investment of five rupees, I made a return of 200% after four days. Now that might seem to those who are not used to options, very incredible, but that's very powerful, of course, often at times, and we will take a small look at it. But key here is the fact that it's a very judged, considered and weighted risk reward ratios people have to take when they do this trading. And it's always a game of probabilities. If you are able to work out strategies by which you will, let's say win 60% of the times and lose 40% of the times and you will win, let's say 1.2X of what you could have on an average lose, then you sort of come out better at the end of the day. And to be able to help a lot of people sort of do it very manually. Algorithms help a lot and that is what I've been working on to do it in the algorithmic way, sorry. Okay. So today's talk we are going to actually, so this is just an example of one of the nifty prices. I mean, the other day, I here didn't include the right hand side, but this almost tripled in price in about one and a half hours. I wish I had included the dub. So you can see at 12.30, it was I think at 89 and by 2 p.m., it had jumped radically. So you could have made 200% returns in 90 minutes or you could have made just a straight 50% return in about 10 minutes, 12 minutes. That's how fast options fluctuate. And which means they also tend to potentially make you lose a lot of your money. And most people start off losing lots of money before they start figuring out ways to make money. But this is just an example of how heavily option prices fluctuate as compared to normal stock prices. Another thing I have to talk about is option chain, but I'll come back to this in a moment. So what we are going to see now is this entire code that I have shared, actually, I will use that link. Let's see. Here's where we are, right? So in Kotlin, what I've done is I deal with different types of sort of scripts. So they might be spot spotters, what your equities normally are, indexes, futures and options. Now here's where Kotlin starts being helpful is in being able to come up with good ways to use generics. So for example, symbol, usually it has a string, but the symbols change in terms of the information that they need to have with them. So if the, an index is also simple, but you can't do much with it, it just got a price. But if it's a tradable symbol, I can trade it. And if I have to trade it, I have to know the lot size, I have to know the tick size, right? So I can start distributing the class hierarchy. I mean, you start using that in the right, and the generics in the right way. So I have a symbol, I have a tradable symbol, which has these added. And this spot symbol, future symbol, option symbol. So, and you can see option symbol has many, many more pieces of information than just ordinary symbol. Because you need that at the end of the day to really understand what is happening with the option. And then what you do is you capture movements over, let's say one minutes or five minutes or 10 minutes and we call it candles. So we come up with a notion called candle. And this candle will have again different information based on what is the type of script. And it will be for a particular symbol. So you could, usually you have open, high, low and close and volume, but index candles, for example, do not have volume associated with them. So I could then have a different interface called volume candle, which I could then use it for everybody else, but the index symbol, but the index candle. So I can start having different types of candles and start bringing in additional attributes as necessary. And you can see now the code is starting to be more and more and more generic heavy, right? And yeah, so you have volume candle and then some candles have what is called open interest information. I think that I won't go into what that is, but if they have, then I create what is called the OI candle. And I declare each of those candles then, index candle, spot candle. Sometimes what I will do is I'll enrich the candles with information that's very expensive to compute so that different strategies will not attempt to recompute that at the same time. One of them is what is called the implied volatility. That's a very heavy mathematical equation. So sometimes, I'll add those. In this case, I think I just added a counter, but yeah. These are just examples of candles. I'll not go into that too much. And this is an example of how you can take an inheritance hierarchy, take generics, and have just the right kind of data structures based on which type of symbol you're using because each different type of symbols have different characteristics. And Kotlin allows you to kind of really have a very sophisticated, strongly type system which will allow you to model your objects correctly in the sense if that object constructs itself, you know, you concentrate correctly. You don't have to worry about nulls and things like that. And so that's one area where Kotlin really helps you. So this is just how the candles and scripts and symbols are modeled. I'll take a pause here and let's check if we have any questions before I come back to this. Okay, we don't seem to be having any questions. So I'll move on to the next step which is where things start getting a little hairy and I'll turn screen share on once again. There's something on chat, let me see. Okay. So now this is where things start getting really complicated. We have something called option chain and this is a live option chain as of yesterday evening. I should have not chosen yesterday evening because that was the time all these options expired which is why you're seeing some of these as 0.05 as the price but for Nifty, this is a snippet of Nifty as of yesterday evening. You can see Nifty will close somewhere between 17, 200 and 17, 250, right? And you had all these options with different strike prices, 17, 200, 17, 150, 17, 100. On the other hand, you had them for 17, 250, 17, 300, 17, 350. And on the right and left-hand side are two different types of options, calls and puts. And again, these are sort of very different types of options. They are very different characteristics. But the key is just for Nifty, if I have to study how Nifty is behaving, I actually have to take this entire snapshot and do a lot of analysis on this to try to understand where are the opportunity areas, what, how Nifty is likely to behave. And this is a snapshot at a particular point in time. Now imagine you start doing this and I create a snapshot once every minute and I store it for the last 15 minutes, then I have to do a complete sort of a three-dimensional analysis across 15 minutes. So that's one dimension, but the two dimensions you see, which is time, but the two dimensions you see here are the strike price and the call and the puts and how they are behaving and so on and so forth. So it's a very complex, very data heavy structure that sort of comes up. And the next thing we are going to take a look at is how Kotlin and its operator functions really help make it very easy to work with these kind of fairly advanced structures. I think the nearest that comes to my mind is the pandas data frame. Obviously it is not more as flexible in that, but it is hopefully the things that I worked on make it so much easier to work with these kind of data heavy models, right? So this is an example of option chain. And remember there is, you know, in the memory what I'm maintaining is a series of such option chains for just one symbol and there are literally thousands of symbols. So it's a lot of data that's sort of getting processed behind the scenes. So there we go. So let's look at what is called the option detail. Option detail is just saying, what is the option I'm studying? What is its ID? What expiry? So an option will have multiple expires, typically 10 to 15 different types of expires, dates of expires. So I know which is the one that I'm studying. Then what is the minimum strike? What is the maximum strike of that option? Because then you need, let me go back here and show you. In this case, this Nifty, the, you know, all the option prices, strike prices are separated by 50 rupees. And so in this case, the span would be 50 rupees. And the minimum and maximum strike would be a long range. In case of Nifty, maybe let's say from 15,000 to 25,000 rupees, right? Or 15,000 to 25,000. And every 50 rupees you will have a strike, a option. And so it kind of tries to capture the important information of how that option is structured, okay? This isn't a runtime data. It's basically reflecting what are all the different types of option prices that are available with that particular option for a given expiry date. What is interesting is this part, which I wanted to focus on, the operator functions get, help you generate some very good and nice access, you know, accessor functions. And I will show these to you in a moment in a test case below. So let me do a time check here, okay? So here's my test case for option details. And I create three, I create options, a variety of options. Actually, no, I create one option detail and then I look at, you know, how many slots it has and things like that. This I'll show you running. So let's run this code. So I have, in this particular case, I happen to be running all the test cases simultaneously. So we will go look at text option details. So I say in this particular case, I have this fictitious option called foo. And it'll have strike prices starting from 50 rupees to 100 rupees. And this is just to check border conditions by 50 rupees to 100 rupees. And at, you know, units of 12.50. So you have, you basically have them at strike prices of 50, 62.50, 75, 87.50 and 100 rupees. So effectively, this will create five different slots. And this part here, like on an option descriptor, if I get it, if I give it a price, it finds me the nearest index slot, which will allow me to look up that particular option details, you know, you just give me a slot number. This is used in subsequent code to then look up further details. But this is, I can take any price and I can just give it in the operator here, get operator and it'll find the nearest option to that price or that strike price. And here it just testing for, you know, am I even returning the right numbers? Another way I can do is, I want to look at a particular strike price and I want to look at a few options above it or below it, or just on one side of it, either all lesser than or greater than. Then what I can do is I can declare operator functions like this. I can say, what is the type of slice I want? What is the starting price? And if I, in this particular case, what I've highlighted is everything less than 70. So it'll give me, if I give it, it tells me I have to look at in the array, eventually slots number two, one and zero. But if it is greater than, I want to use two, three and four. So these are all different type of, you know, a very different way of accessing what are frequently used mechanisms to sort of subset an entire option chain. And this kind of operator functions really make it very easy and useful to then quickly narrow down the subset. The whole data is stored in the array. So these give you the array offsets very rarely, which this particular class called option chain, then uses very well. I won't go into the details of option chain right now, but I will show you how it gets used in a test case. So here is just a simple option chain construction. So here it showed me, this was a poor man's implementation of a option chain printout, which is what you were seeing here, right? So if I want to print it out, I can get to see an option chain very easily. Then you start, you know, figuring out important ways to access the option change along certain dimensions. As we saw, it's a three dimensional array really. And sometimes you want to cut across time. Sometimes you want to cut across different strike prices at the same time. And you can, I think Gotlin really gives you a good set of capabilities to be able to structure your functions just right. So here I'm saying, for this operation, give me the PCR. PCR is a domain term called put call ratio. So I want to compute the total number of, the ratio of the total puts and total calls. And there is a whole bunch of stuff about, how to interpret that, which is kind of separate. But let's go here to this function. And here I can say get puts and, you know, map not null because sometimes you can get a null, though most of the times you will never get a null, but you just want to make sure you ignore a null if it comes. You do a sum of all the open interests for those puts and convert them into a float. Similarly, take all the calls and just create a from the calls extract the UI, which is open interest and sum them across all different strikes and divide one by the other. You get a put call ratio. Similarly, you have things like CUI, which is combined open interest, where I can say for a particular price, right? And a particular style of slicing it, I mean, around it. What you're interested in is right now the price is 37. What are the five strikes just around 37? Two before, two after and one that nearest. And take them all and then add up the open interest. So when you want to do that, again, you can, the slice operator is again, a helpful operator that I wrote here. And then you can, you know, do all the operations on it using, you know, essentially sum or whatever. You'll consistently see map not null, but that's just because I have to deal with the official possibility that sometimes some of them may be nulls. My internet connection might have problems or whatever. So it just so that the code doesn't break completely. So you can, this is again, a lot of capabilities you get to be able to structure your code in such an accessible way. And when you put it together like this, you get a lot of capabilities to really be able to write functions very succinctly. So here I'm saying, you know, for a particular option, for a put around the price of 250, five is the span. So here two before, two after and one nearest 250 and add up their put OI and give me that. And test cases, the expected value happens to be 240. So that's one way to kind of, you know, do that. Sometimes you just want to look at the greater than, I mean, just one side of the price, depending on what analysis you're doing. So on an option chain, you can, you can build a whole range of this kind of capabilities. And finally option trail is a term I came up with, which is an array of option chains over time. And again, I won't go into the code in detail, but we will look at in the test case, how it gets used. I have only one example there, which is to say, if I have option trail, I want the last three time stamps. So let's say right now, one minute ago and the one before that. I want to extract all the calls and on each of those calls, I want to compute the CUI and give me the list, right? And I'm saying, okay, in this case, I expect the values to be 386, 399 and 412. Now there's a lot of intelligence you can do, apply to try to understand, is this bearish, is this bullish, what is the current trend and stuff like that. But the interesting thing here is again, I was able to kind of very succinctly create a mechanism to extract the right kind of information I want. So it's, for those of you who are familiar with pandas data frame, this is somewhat very crudely similar, but I think one of the things that Kotlin really helped me do was be able to write functions which are very easy to understand and one doesn't get bogged down in the details of how some of these access data access really works. So that is how you can take fairly sophisticated and large data structures and yet do flexible analysis at one time and fairly rapidly, I must say, I literally end up recalculating the market every few seconds and that's a lot of work. So this is very helpful. And if I'd done it in a different language, some of them would have resulted in much, much larger pieces of code and much with so much more accidentally complexity in them than what Kotlin allows me to get away with. So that was something very pleasant here. So yeah, so this is my, this includes the second part. Let me see if there are any questions. Now's a good time to ask any questions if you have any. The third part we are going to get into is really the actors and concurrency. There's no questions at the moment. Sorry? No questions. There's no questions. Right, okay. Okay, so this was the other part. And the third part is the core routines and here's where we will take a look at, what does it take to sometimes build algorithm training systems in terms of the architecture? And here I have kind of, you know, relatively simple one model I created. What you do is you get from an exchange data feed, you get an exchange data feed from a data feed vendor and you get like, I don't know, 500,000 ticks every second. And then what you do is you try to break them and combine them into candles, which you might, depending on how you want to trade group it every 15 seconds, one minute, five minutes, sometimes even hours, and then, you know, turn out and spit out those candles. And those in turn go to your strategies. Now, any algorithmic trader would actually focus a lot of time in the strategies, but that's the one thing that I have kind of completely bypassed here because that gets into a very, very financial domain. But what is important is all the strategies that are doing their own stuff almost independently, but you need to be keeping track of a lot of stuff simultaneously. And a lot of this communication is asynchronous. So A, if different strategies are yielding different trades, you need a global view to take a look at things and say yes in this particular case, maybe three strategies have all suggested buying the same script and you don't want to, you know, overload your risk on that just one script. So you say, okay, the risk manager might reject two of them and say, sorry, we already have a position in this, or you don't, you know, although the strategy suggests we take up exposure, we are not going to take an exposure and so on and so forth. And if the risk manager approves it, it will go to the trade manager. And what the trade manager will do is it will basically place the trade, but again, it is very asynchronous. So once it places a trade, the broker API is going to turn around and maybe sometimes take even minutes, if not seconds to return saying yes, this trade actually did get placed successfully or I couldn't do it or I had an error. And those events are, you know, a lot of time apart. Meanwhile, strategies are continuously running every minute or every 15 seconds. Or even sometimes there are strategies which will run only once a day, so it depends. But you can't have the strategies wait for the trade manager to block on a call that was made to broker API, which took five minutes to fulfill. So what we have to be able to do their work asynchronously. So you need a synchronous mechanism of communication between all these components. And that's what we are going to take a look. And these are just some of them. I mean, the actual architecture gets a lot heavier at times, like these are the more important ones. And then you obviously need a monitor. I've skipped the monitor part of it. So, and even the risk manager, I just kept accumulator, trade manager, strategies and see how Kotlin channels and Kotlin actors help us do that, right? So that is the next phase. Actually, let me go back here. So I say, okay, I declare some types, which I before, yeah, here's where I need to start. So let's say I have a symbol foo again, and that's the one that I'm going to use. And I need an aggregator to sort of accumulator to keep on taking ticks and aggregating them and then spitting out, let's say one every minute. Obviously, that's not the work I'm doing here. It's a much more simplification. What it does is it just keeps on sending out candles at no predefined interval, right? I've made the interval here a little, I mean, configurable, but this produce function actually creates a channel and then returns that channel to you. And if you look at the type of this produce and this function, it is returning a receive channel, which means this will create a channel. It'll keep on pushing data to that channel and it's given you a handle to the receive channel, which means on the receiving side of things, you can take it and take data and read from that particular channel. So this is where candles are sort of getting generated. Then there is one more called enricher, which takes that receive channel of a spot candle and returns another channel, receive channel, which will be on an enriched spot candle. So you sort of changed the data structure a little bit, added a few more things in there, which is what is often necessary. Then you took a, sorry, I don't remember what aggregator was. Aggregator was the one where it took many candles and aggregated them into a single candle and you're going to eventually the strategy is going to spew out an order. So you need an order type, whether the order is for buy or sell, you need order status, which says whether it was initiated, approved, placed, partially fulfilled, filled, aborted, canceled, so on and so forth. So you need to keep track of each order status and this is the order. So you have a strategy ID, what is the order type, what is the symbol, how many did you order, how many have been fulfilled and so on and so forth. And eventually the trade manager is going to return with the order response. Saying this order ID, so you might have given an order for 20 lots and it turns around and say, okay, out of 20, I have filled five. And then it will come back and once again say, okay, out of 20, I have filled seven more. And so, and maybe the rest of them don't ever get filled. So the trade manager always has to keep on returning order responses. And this is just the interface of a trade manager and this is a implementation called a paper trade manager. What it does right now is this very simple logic. Moment it gets the order, it turns around and returns the order response which is going to, which is going to say, I filled everything. But interestingly, this here is going to turn around and actually not return a value, it's going to look up the underlying strategies which plays the order, right? And then send it a message on the channel saying I fulfilled your order, I didn't fulfill your order. So the trade manager can asynchronously and directly communicate with each of the individual strategies that are running. Similarly, you have a strategy and the strategy may need to process either a normal message which is your candles that are coming in or sometimes it may need to process an order response. Saying this order got fulfilled or did not get fulfilled so that they can keep track of things correctly. Again, I have two very, very simple strategies. And right now, as soon as the message comes in, the channel comes in, it just places an order and if it gets order response, it just prints it on the screen and both strategies are identical. But what is interesting is I turn around and wrap the strategy in a strategy actor. So this is the actor implementation and I give the strategy a reference to the trade manager actor. And what I am able to do is, I take every message as it's coming in which is mostly the candles, I process it and if the process method says, okay, now I need you to issue an order, it will send a message to the trade manager actor saying go process this order. If the message happens to be an order response that the trade manager sent it, it will call the process order response method. If it is neither of these two, it will say, okay, this is some message, I can't process, right? So that is what it is doing. So this is all taking normal objects and then wrapping them up either as objects which send messages, which are producers or consume messages, which are actors. And you can do all that stuff with the various concepts that Kotlin helps you with. And it's sort of a language kind of model. Obviously, JVM is different than Veeam, so it doesn't have the same kind of memory isolation and everything. So you need to be a little bit more careful. You could have the same object in multiple actors and you will obviously, because they are in the same JVM, same memory space and like in Erlang, you will be actually be simultaneously working on the same object without any safety from concurrency. So if you actually end up accidentally creating something like that, you could be in trouble. So either you have to protect your objects from simultaneous access from multiple chat actors or you just make sure that every object has just one set of actors around it, whichever way you want to look at it. And this is how this particular, what I've done is I've here set up a sort of sample, I have wired up all the components today together. So initially there is a component which kind of creates candles, then another one, and these are all independent sort of actors. So they are all communicating with each other asynchronously. So candles is your candle producer. Enricher is the one which is enriching those candles and then giving out enriched candles. Aggregator kind of aggregates multiple candles into one. So groups them together. And then I start creating strategies. So I have to create a empty, this is one of the places where you sort of do a mutable map because I need to give the reference of strategy actors to the trade manager. So I'll fill up the map later and it's, and then I instantiate a paper, a trade manager actor. So now it's going to do the trade management. And then I instantiate each strategies. Now these strategies themselves are normal OOP objects but that I turn them around and make them into actors using the strategy actor function, right? Which if you go there, it basically wraps it up, wraps up the object with a kind of actor implementation. And so now I've created an actor for each strategy. I decide these are all the channels I will need to close when the program ends. And then I get this data and send it further down. And you will see here, what test case is this? This is test option, yeah, test. Here in test basic, I should have given it a different name but you can see all the actors printing out those messages saying, I received this stick, I despised this stick. The strategy says, okay, I got this strategy. Who says I got this? Trade manager says, okay, I got this order and strategy then gets an order response and so on and so forth. And it puts it all together. And you can see all those different actors really working independently or synchronously, concurrently and yet you don't have to worry too much about trade safety and things like that because you kept everything isolated and thus, and you're not blocking anywhere. Most importantly, I mean all the HTTP calls end up sometimes taking a long time and sometimes even with the calls at Siege, the response is going to come much later of when a trade will get fulfilled. So, which could be minutes away. So this allows you to implement a lot of concurrency altogether at the same time. So yeah, I think this sort of was what I wanted to talk about. I know it's a little bit dense, but I left all the code out there for you to take a look at. But yeah, I'm open for questions. Dhanje, thank you so much for sharing. It was dense, but there was lots in there. I'm sure that that'll be of benefit for somebody just seeing how you've teased out the ideas and what you were trying to achieve and what you've been able to do using Kotlin. So thank you for sharing your experience.