 Hi everyone. Thank you for joining this session. I'm very happy to be here in person in London and seeing everyone in, you know, live 3D, like everyone mentioned. And so this is really great, open source finance forum. I think it's very important. And I'm here to talk about one of my favorite topics, the challenges of deploying real-time AI for finance and how open source software and tools can help. And I'll be focusing specifically on feature stores. So I'm Nava Livy. I'm an AI dev advocate at Redis. Redis is the super fast open source real-time database used for real-time applications and use cases. And we are the home of the open source Redis as well as Redis Stack and Redis Enterprise, which extends Redis to, you know, enterprise tiers and capabilities. And I'll talk a little bit about that as well. So we'll start off by the challenges of deploying real-time AI for finance. What are machine learning operations and specifically what are feature stores and for real-time AI? And towards the end, I'll go through a few case studies specifically in FinTech, in finance and banks using open source tools to deploy real-time AI and as well as how to get started with two great open source feature stores, Feast and Feather. With Redis as the online feature store. And I'll explain what is online feature store and what are feature stores, et cetera. Okay. So I think the good news is that the adoption of AI in finance is, you know, is growing. And this is from a study by KPMG that shows that a year over year there was a 37 percent point increase for adoption of AI in finance. And this is much more, for instance, if we compare it to the tech sector, which only had a 20 percent point increase year over year. So it shows that that really finance are, you know, getting excited about the value of AI and machine learning. And when we look at, you know, the use cases that banks and financial services companies are deploying, then we can see, you know, use cases that are more specific to the finance domain, like fraud detection and loan approval and anti-money laundering. And, but we can also see more generic application to use cases like chatbots and recommendation and lead scoring, which is important for any business that is involved with digital transformation. And all of this is really driven by digital transformation, by disruption from digital natives, by the low interest rates, by the threats coming from cyber criminals who themselves are using AI and machine learning, and ultimately the need to improve customer experience while not, you know, keeping the cost low, at least doing it cost effectively. And over time we see that those use cases are becoming real time. And that means that they have to be deployed in a way that the response is instant, without the customer even being aware that there is a machine learning engine running in the background. And deploying these machine learning use cases to production is very hard and complex. Doing it for real time use cases is even harder. And this is why we see that most models, machine learning models, never make it to production. And some that do make it to production after a while, they crash and burn because of all kinds of issues. And this led to the development of the domain that is called machine learning operations or platforms, MLOPS platforms, that basically bridge the gap between what the data scientist develops on his Jupyter notebook, okay, the experiments that he creates, between what he takes to deliver a system, AI based system to production. And there is a huge difference, okay? And on the heart of this machine learning operations platform or the cornerstones are what is called feature stores, okay? So it's a component that is becoming more and more important. And I'll explain what they are in a little while. In this talk I'm going to be focusing on two open source feature stores, or more likely two open source feature stores, Feast and Feather and Redis as the online feature store. And the online feature store, as you will see, is the critical component for delivering real time AI. Okay? The feature store is very important for a lot of things and we will cover them. But the most important component for real time AI is the online feature store, which is, you can see in both of them, is with Redis. Now the Feather feature store is a feature store that came from LinkedIn just a few months ago. LinkedIn decided to open source the battle tested feature store after many years. And you know, it's for us to use it. And Gorjek already in the beginning of 2019 decided to open source their feature store. LinkedIn did it with the collaboration of Azure and Gorjek with the collaboration of Google. And we have Feast to use. And this is the most popular open source feature store today, Feast. And it's interesting because I don't know how many familiar with Gorjek. LinkedIn I assume all of you are familiar with, but Gorjek is like the Indonesian Uber. I saw a market cap from a year ago of $10 billion market cap. So it's a huge company. But back in January 2019, it was a much smaller company, as you can imagine. And then it used the Redis open source. But only recently due to the scale that they have to operate and managing the scale and managing it cost effectively, they decided that they need, you know, they outgrown the Redis open source. And they need the Redis enterprise. And this is just to get somebody asked me, does Redis open source can comply with enterprise capabilities? Yes, with Redis enterprise, we can. And it's the finance vertical and telco and government and healthcare are some of the biggest vertical industries for Redis enterprise specifically because of those reasons. And when you look at those, you know, those architectures, you can see lots of logos. And a lot of them are open source software and tool, right? But this is really just a fraction, a very, very small fraction of the open source tools after that exist for machine learning and AI. And when we look at them, they cover every category and every stage of the life cycle of machine learning project, from data management, to training the model, to deploying the model, to monitoring the model, maintaining the model, and really every aspect of it. And this is also just a small fraction. There are thousands of such open source software and tools. And of course, not all open source software and tool are alike. So it's important to make sure, you know, what is the popularity of the tool? Does it have enterprise support? And many other aspects. But when we look at open source software, for machine learning and AI, we see that for every category, if you look which is the leader, which is the best of breed for every category, most of the time it will be an open source tool. So basically, we won't have seen the innovation that we see today in machine learning and AI without open source. Okay? The researchers, companies, won't have been able to reach, you know, the amazing breakthroughs that they have, you know, that were done. And the value without open source software and tools. And as I said, we're going to focus on FIST, Redis, and Feather. And you can see the cards here of each of them. I just want to mention that FIST is also a Linux foundation project. It's specifically a Linux foundation AI and data foundation project. Now, when we look at what problem, what problem or challenges do feature store address, okay? So what they address is the most difficult problem or the most time-consuming problem that data scientists have to tackle with, and it is the data management, okay? According to many surveys, more than 50% of the time that a data scientist spends is about data management. It's cleaning the data. It's transforming the data. It's feature engineering, okay? And this is something that, you know, this is, you can see here, 66%. I've heard the number, 80% thrown around. It doesn't matter. It's more than 50% of the time. And according to many companies, this is one of the most crucial bottlenecks for productionizing machine learning models. So feature stores deal with this problem. And when we look at what our feature store and how do they do that, so here you can see a very simplistic view where the feature store sits. So there is the raw data, and there is the model. And in between, there is the feature store. And the model can be for predictions or for training. And between the raw data and the feature store, there is the feature engineering. And just if anyone here is not familiar, features are just, you know, a fancy way of saying data inputs for a machine learning engine to consume, okay? So if we have the year that I was born, okay, then a transformed feature would be my age, okay? For instance, this is very simple. Or it could be the last transaction. So a transformed feature could be the average size of a transaction, okay? And the feature could be, it could be a categorical feature, like maybe what nationality I am, okay? It could be numerical. It could be an embedding, a vector embedding, a learned vector embedding from deep learning machine learning model. And it could be even an API, okay? And so this is where the feature store, what the feature store is. And then you need to extract those features for model prediction and for training. And for real-time application, you need to extract those features very, very fast. So the challenge is that feature store address is serving features for real-time prediction with low latency. And this is the most important challenge that feature store address in terms of real-time applications. Without it, there isn't a real-time application, okay? It just isn't. Then the second challenge, which is also very important, is avoiding training service queue. What happens is that we train the model, we reach a level of accuracy, and when we deploy to production, there is often a deterioration in the accuracy. So if we reach 95%, it could be 85%. Now, not all of it is because of, you know, the difference between training and serving, okay? Some of it is because the data is different, but anything that it has to do with, we transform differently the features during the training pipeline compared to how we did it in the serving pipeline. So slight difference is something that we want to address with the feature store. Then there is the issue of, if I develop the feature for one use case, then there shouldn't be a reason why another data scientist in my company should reinvent the wheel and develop, you know, the same feature for his use case. Why not re-share features across use cases? So this is the third aspect that the feature store is very important, and that helps solve the 80% time or the 66% time that data science spends on data management. And the first aspect of feature store, which is more recently added, is the monitoring data pipeline. So this is very important also for explainability, for observability, for regulations, for compliance, okay? And especially important for the fintech, for the financial services domain. And because we are talking now about the challenges of deploying real-time AI application, we are focused on the first challenge, which is serving features of real-time predictions with low latency. So what do we mean by real-time? Okay? So we mean that the serving with the prediction will be upon receiving an online request. It could be as the customer is doing a search, or as there is some kind of interaction with an application or website. It has to be not just timely, okay? It has to be based on live, fresh real-time data. So not the data from last night batch processing, and not even from an hour ago. Live, fresh data, I imagine, solving, you know, identifying fraud detection, if it's the data of the last transaction size, the last three transactions, you know, from just 20 minutes ago, compared to something that happened maybe five hours ago, or maybe just from the last few seconds. So live, fresh real-time data. And doing it extremely fast with a very low latency, okay? So measured usually in milliseconds. And this has to be done consistently. So think P99, okay? And as I said, over the years, we see that the use cases are becoming more and more real-time. So the response time is expected to be, you know, faster, or the window is smaller. So if in the past I worked in a fraud detection company, then it was 10 years ago, 24 hours was acceptable. And batch processing without any machine learning involved was also what we used to do. Then three, five years ago, we introduced machine learning and the window 40 minutes. And this became even smaller, you know, to 12 seconds. And now it's 50 to 200 milliseconds. This is basically the expectation. And the reason is that you need to stop fraud as part of a transaction. And a customer isn't going to wait for the transaction to be executed. He expects instant response. And if there is a problem with the machine learning engine that is running in the background, okay, then either it will give a false positive or false negative, which is not good. You know, money is going, maybe lost. Order is going to be a bad customer experience because he didn't yet get his transaction approved or vice versa. So this is why it's so important to do it consistently and in real time. Now, this end-to-end latency is made of a number of components, the network latency, the application pipeline, maybe it's the e-commerce application, serving the features, okay, extracting those features and feeding it to machine learning model and the model scoring, okay. So all of it has to be less than the end-to-end 200 milliseconds or 100 milliseconds, whatever the case is. And this is very, very difficult. If we look at the pipeline for real-time AIMM, there are lots of places, lots of the choices that we can make that affect these end-to-end latencies. So what are we using for the feature engineering, you know, from the streaming sources or from the batch sources? What are we using for the computation of the streaming? For the feature serving, what method are we extracting those features? Are we using Lambda function? Are we using an HTTP server, a Python HTTP server? Are we using a Java GRPC server? For instance, our friends at FIST, the open source feature store that I told you, they found that there is a huge difference between using a Java GRPC server and an HTTP Python server. And while for data scientists, it's much more convenient for us to write in Python, Java and Go often will be faster, more efficient. So you need to check what is the best for your case. Also, in the serving part, we can optimize. We can use hardware accelerators, GPUs, TPUs. What serving engine are we using? Maybe we can make the model smaller, so it will be more efficient. But what companies found that the most important component to optimize and the biggest source of bottleneck is the feature store. And the store that stores those features or data inputs for the machine learning model for inference. And this is why it's implemented with an in-memory database compared to a disk-based HMS. And often for high-scale or most often for high-scale or low-latency use cases, Redis is the online feature store that is selected for those feature stores. And the reason for it is basically benchmark, benchmark, benchmarks. Because it's such a critical component, companies before choosing the online feature store, they do throw benchmarks. And sometimes they share those benchmarks with the community. And this way, we can also enjoy the benefit of their benchmark and their work. And this is really a nice thing about the machine learning community in general. And so you can see here benchmarks from Uber and from DoorDash and from Feast Open Source and Tecton, the commercial feature store. I'm not going to go on every single one of them, but there are links to all the benchmarks. But I think the key takeaway is that Redis is by far the most performance in terms of throughput and latency. And not only that, it's the most cost-effective. And I think this is something that isn't very intuitive to many people because memory is more expensive than disks. How come Redis is more cost-effective compared to, let's say, DynamoDB here? It's 14 times less expensive according to Tecton. Redis Enterprise compared to DynamoDB is 14 times less expensive at the same time, much faster. So the reason is that it's just a much better architecture for it. And here we have a cash up, block cash up. I don't know if you're familiar with it from the Fintech, a very, very successful startup from the Fintech board. And this is just from a few months ago. It's a very good talk. They're explaining how by moving from DynamoDB to Redis, they shifted the entire latency curve. So they went from 30 milliseconds to 10 milliseconds. And I encourage you to watch the video. You will see how every latency, every millisecond of latency counts. And anything that can be done to optimize is being done. And here at DoorDash, they explain how using Redis, they moved from different data structure. And by that, they reduced the latency. So Redis isn't just a key value store. It's a key data structure store. Any data structure that you can think of, JSON, CRAF, DIME series, can be implemented with Redis. So it's not just improved the developer experience, okay, developer productivity. It also has an effect, an impact on reducing cost, improving performance. So here they moved from a flat key value pairs to Redis hashes. And they improved the read latency and the memory footprint by 40%. And a CPU efficiency by 5x. And this translates to money. And why is it important? The online feature store for high-scale use cases can be more than 50% of the total cost of the machine learning operations platform. So any savings you can do with your feature store is really beneficial. If you have a few users and the use case isn't very successful, then it doesn't really matter. But for when those use cases become successful, and that's what usually happens, then the data sets become larger, the full puts, the number of transactions per seconds, or reads per seconds, or queries per seconds, you know, go into the millions. And then the cost is much larger and optimization is important. Now, up till now, I talked about Redis open source, okay. And Redis open source, 90% or more of the organizations use Redis open source. And they enjoy all this goodness. So if it's the scale, the low latency, high throughput, the persistency, okay. And the different data types. But so this is really good. But for telcos, for finance vertical, for governments, for healthcare, this isn't enough. We all know it's not enough. And also for other verticals, when they become a very large scale, like the Gojek example that I gave you, when there were a startup, they could do with Redis open source. But now that they are much bigger and much more successful, they don't want to manage the Redis clusters themselves. They move to Redis enterprise. And another customer, the use case became so successful, they don't, they wanted to save 50% of the cost. They went to Redis enterprise. And another bank, they wanted something to have five nine availability, three nine availability, wasn't good enough. They wanted the security, and all the things that Redis enterprise can give. So they went with Redis enterprise. Okay, so here you can see what Redis enterprise gives on top of Redis open source. And I think that it was mentioned earlier in the keynote, the importance of C O S S commercial open source software. So it's great. Open source is great. But being stuck with a great open source technology, but without 24 seven support, and, you know, waiting in the forums or stack overflows that somebody might respond is not something that is good for every company. Okay. And so all those extra, you know, layers that that enterprise need are provided by Redis enterprise. Now, I just want to focus about reducing cost. So Redis on flesh is something that we see often is used for very large scale feature stores, you know, in the terabytes of data for the feature stores, because it allows to reduce the cost even further. All the benchmarks that I showed you before are without Redis on flesh. But when you have your data, your feature story becomes even larger and larger. Then you might want to go with Redis on flesh. It's a tiered memory access. So the keys are in the DRAM and the hot values are in the DRAM in the memory. But the warm values or the warm features are in the flesh. And this way you can reduce substantially the cost, but without impacting performance. And this is something that isn't the expertise of Redis. It's very difficult to do. We know other companies who tried to do it and didn't succeed. So, and this is something that we see more and more feature stores for high scale are using. Applty now I talked about the real time aspect of a feature store. I want to just touch a little bit so you can understand why feature store is becoming such an important component regardless of the online feature store. So if in the past a feature store could have been just the online feature store without anything else, today we see a feature store has two manifestations. One the offline store and one is the online store. And the offline store is used for training and for batch predictions. And they, before we saw the example of DoorDash they used snowflake for the offline store, but it could be Redshift, it could be S3, it could be SQL Azure, it could be Google BigQuery. And in the offline store we store historical data and static data. And the only online store we store the latest data. And this is, as I said, the critical component. And on top of the storage and materialization layer where we store the features, there is the management and orchestration layer. And because of the management orchestration layer we can achieve all these other challenges that I mentioned earlier. So how do we avoid the training and serving skills, those little differences between how we define the training pipeline and the serving pipeline is we define all these things in the registry on a logical level so we don't have to translate from Python to Java. It's all stored on a logical level and on an abstract level and then you can more uniformly execute it for the training and for the pipeline. Now once the features and the transformation logic of the features is stored in the registry on the logical level in the feature store, then it can be reused not just across training and production, it can also be used across another use case by the data scientist that created those features but also by another data scientist in the organization. So this is how we avoid what used to be the best practice up until a few years ago that each data scientist in his own silo creating features for his use cases and not even being aware that another data scientist in the organization is creating those same features. And even if he was aware, there wasn't really an easy way of just sharing features across the organization. And finally there is the monitoring of the data pipelines. So how can we monitor? For instance, we can monitor for the volume. We can monitor for latency. So these are like the operational aspects. And we can also monitor the data scientist aspect in terms of accuracy. For instance, the distribution, the distribution of the data that is coming into the features, into the feature store and the distributions of the features that are being served in predictions. If the distribution changed, something happened, okay, maybe there is a concept of the data, it may be another issue. We have to investigate, maybe we need to retrain, okay, or some other thing. And we also need to make sure that the freshest data is being served to the feature store. So all of this we can monitor with the feature store and not all feature stores have it yet. You know, this is like something more, it's more new. Now, why is it so important? In AI, and especially for finance is going to be very highly regulated. And there are lots of new frameworks, regulations around AI, and specifically in finance, the demand that there will be monitoring and logging and explainability and make sure there is no bias. Why? Because finance, maybe for financial vertical, maybe more than any vertical, any decision that an AI makes can have very big impact for good or for bad on the people, okay. You can't get a loan. You can't get credit card. This transaction is fraud and it's not, okay. So this is why there is a demand, more and more demand for for AI to be governed. And so this is the example of the EU AI Act. They define it's a risk-based approach. So high risk are applications that banks do, underwriting of loans, fraud detection, transactions, not just that, okay, they're not picking on finance, they're also others, but definitely those. Now, chatbots in banks, they will be under low risk. And the only observability, if you start talking to a chatbot, the chatbot would say, hey, I'm a chatbot. I'm not real human. So that's it. Just transparency. That's it. And then there is also project fair. We are here in the UK. So I don't know if you're familiar with it. So project fair is in partnership between HSBC and the Allen Turing Institute. And fair is a framework for responsible adoption of AI in finance, specifically in finance. I think it's very important because the more we have such initiatives specific to finance and more companies will be involved from the finance world, it's the same as open source. It will be, it will serve everyone, right? And when we look at the feature store and what are the expected, this is just a draft, right? What are the expected requirements, then we can see that feature store really can help in every, almost every aspect of those requirements. Now, I wanted to touch a little bit about case studies of online feature store with Redis and with Feast and Feather. So in general, when we look at, and on the left, you can see companies using Redis as the online feature store. Most of them are not using open source feature store. They're not using commercial feature store. They're using do-it-yourself feature store. They developed their own feature store and they're using Redis as the online feature store. But this is changing. More and more companies are now adopting open source. And I expect that in the future, more and more companies, smaller, you know, small medium enterprise will adopt commercial open source feature stores. And Redis is, you know, often deployed in any of those cases. And when we look at, specifically at FinTech companies, then I put dollar sign, okay, next to the companies that are from the FinTech. And there are also two banks here that, you know, banks are more shy, okay, so I didn't put the logos. And which ones are using Feast? I put also here. You can see which ones are using Feast. The reason we don't have Feather is Feather is really just a very new open source feature store. It's an old feature store at LinkedIn, but it's a newly open source feature store. And now I think it has just over 800 stars and Feast has 3,000 stars, but it's growing very, very fast. So I would follow Feather closely. They have some advantages that Feast doesn't have. Both are great open source feature stores. And here I wanted to focus a little bit about use cases that are used for, specifically in the banks and financial services. So we have online mortgages companies better, which is, you know, a very popular company, a cash app that we talked before of Block, a very large bank, very, very large. And Robinhood, which is also, I think, commission-free stock trading. It's also very well known. And you can just give you an idea of those scale. So online market is better.com. They have almost a billion dollars in revenues. They grew 10x a year prior. I think this is really important. When we look and read this, we are in advantage points to look at those companies. They are growing very, very fast. They're using machine learning. And the fact that their online store grows very fast means that their machine learning use case is very successful. So they grew 10x a year prior. Mobile payment service cash app of Block, what was called Square App. 70 million annual transaction users, 1.2 billion gross profit. Now, you might say, ah, compared to banks, this is nothing. But remember that this is just one service. Okay? It's not all the service. And it's going very fast, right? So a very, very large bank, I think top five. Okay? And Robinhood, 30 million users, 2 billion of revenues. So all of them are using. Real-time AI, it's scale. Okay? And very successfully. And they're using online feature store Redis. And two of them are using FIST for the open source feature store. It's not do it yourself feature store. Now, the bank specifically here, you can see that it says enterprise. It means that they are using Redis enterprise. Sometimes companies start with Redis open source. They outgrow Redis open source. And then they go to Redis enterprise, like the cases of Gojek. And sometimes they go directly to Redis enterprise, let's say from a slow disk based database like DynamoDB in the case of this very large bank or for, you know, just, you know, from nowhere. They just, you know, have a use case and they're looking for the best option, the best of breed option. So these are real-time high-scale use cases here. There are links and I'm also going to share all the links in the recording on YouTube. Okay? But you can see here in the architecture that the data sources are many, many data sources to the feature store. There are several offline feature stores and several streaming sources and they're using TTL, Time to Live, which is a native Redis capability, to make sure that the streaming features that populate the online feature store don't just clog it, you know, and they will just expire after 24 hours or 48 hours. This is a lead-scoring use case, okay? And the streaming sources are supported for FIST only for Redis and it's very important and he explained to it very well why streaming is important. I encourage you to look at it. Then cash up again, there is a source of it. I already talked about it, how they shifted the latency curve by moving away from DynamoDB and this is for personalized search and recommendation. And then we have Robinhood using for fraud detection and Robinhood are calling the feature store beast, not FIST, beast, not because it's a monster, but because it's built, built, I guess it was an internal joke, but it's built with FIST and they explained very nicely how for some things FIST is great and they adopted it, for some things they didn't like how FIST was implemented, they didn't take it, some things FIST was missing. For instance, transformation logic and actually executing those transformation, the feature engineering can't be done today in FIST, tomorrow maybe, but not today. In Feather, you can do. I told you I will tell you that there are some advantages of Feather. Feather is more, it wasn't open source in the beginning of 2019, it was open source today, so it managed to do much more improvements along the time and they're using here Hive as the offline store, so offline store can be many options and here the bank, it explains again why they moved away from MongoDB to Redis, to improve the latency, to reduce cost, and you can see here the architecture and they're using Redis Enterprise for fraud detection and balance forecasting and it's, you know, it was very interesting to listen to the customer because the way he was talking about fraud detection and the way he was talking about balance forecasting, so you have to, not all use cases are alike and fraud detection is very mission critical and it always is very important, you know, five line availability is active, active, everything, nothing lost. Balance forecasting is also very important, this is how many, what is the reserves in terms of frequent flyer miles of the credit cards to keep and, you know, not to have too much, but not to have too little is important, but it doesn't directly affect the customer, so when anything that affects directly the customer, it's very important that it's airtight everything and one of the reasons, you know, coming to Redis Enterprise, now here I'm going to point you to a few links for tutorials and resources, how you can get started with FIST and Feather, both of them are deployed on Azure also integrated into data and AI ecosystem of Azure, so it's very convenient if you're already using Azure, then you can use Feather or FIST on Azure with Redis with Enterprise tiers, okay, so Microsoft did, you know, a wonderful job in it, I told you the difference between Feather and FIST, the main difference is the transformation logic and the feature engineering and so I'm going to share a few links and resources, so for FIST, because everything is open source, you know, the FIST Feather Redis, then if it was Zoom, I would say 10 minutes after this talk, but we're all in London, so when you get home wait, you know, say hi to your kids and family and then you can just start playing around with FIST and Redis or Feather, so we created a tutorial, some people prefer to use Collab, so it's also available in Collab, and you could just, you know, see what is a feature store is all about, and what is FIST is all about, but generally if you want to get more familiar with the concept of a feature store, I think this is a very nice way to do it, and then you can say, okay, it meets my requirements, it doesn't meet my requirements, but it's very easy to do it, and this is FIST deployed on Azure, you know, into the AI and data ecosystem of the Azure platform, you can enjoy the enterprise tiers of Redis by including Redis on Flash, it's called, on Microsoft, it's called Enterprise Flash Tier, so all the goodness that we just talked about is available on Azure, okay, and there is here also a link to a tutorial specifically for FIST on Azure, so you can also do that, and then there is Feather, so FIST is much more lightweight, okay, so it has, so even if it's, like if you were a small startup and there is only one data scientist, I would say go for FIST, check it out, because it's so lightweight, Feather is a more robust feature store, but still there are great tutorials, there is a slack channel for both FIST and Feather, and I shared before the FIST one, this is the Feather one, and again, there is a very nice quick start guide, how to start with Feather on Azure, there is a very active slack, that if you have any issues, you can just post your comment there, and everyone will be happy to assist, and so the last slide, okay, always hate the end on a high note, so this is, also before it was a high note, okay, so this is from the survey, Fino survey, and they asked, they asked customers what is the most beneficial area for open source engagement in finance, specifically in finance, and the answer was, there are rules, okay, the answer was that AIML is number three, so you know I'm very happy about it, it means that there is a bright future for open source in AIML in finance, and as good as it is to consume open source, right, I just told you great ways of how you can consume redis, as online store FIST and Feather, I think it's also very nice and very gratifying to contribute back, so for instance, FIST had an issue with the flow, the real-time flow, and even though it wasn't related to the integration with redis, because we are, you know, we know everything about real-time, we helped FIST improve the flow, you know, by 30%, okay, this is our expertise, now your expertise is finance, and I'm sure that you will see that FIST and Feather is lacking in specific ways that are important for the finance domain, and so instead of just developing it for yourself, you can develop for yourself, and then contribute it back, and everyone will benefit, now if everyone does it, okay, if anyone does it, then finance will move forward in the same rate as AI and machine learning will move forward, right, so I think this is, open source is great, of course you have to do it carefully and securely, and I explained how redis enterprise can help, and I think also a partner like Azure and FIST and Feather is available in Azure is also a great way of reducing risk, and that's it, so I look forward to seeing your contributions, thank you very much.