 Good afternoon. Thank you for taking time for going through our presentation. This is something we have done in Capitlone. And that is something we wanted to share with the audience as deep dive on partners redemption capability using the serverless. I hope you guys will have some takeaways from the session before we really dive into the details, some details on Capitlone. Capitlone is the first US bank to exit out of legacy on-premise data centers and go all in cloud. So you can imagine how much of a cultural change or technological transformation a company would have gone through to achieve such a feat. So right now, we think we are a technology company which happens to be in a banking business. And we are still a founder-led company which stays true to its mission, which is change banking for good. There are a lot of things in which we do that. We give back to the community. We do participate in a lot of community events. But when it comes to the tech, we do that in the form of open source. So we have contributed a lot of projects as open source projects, which came from our own implementations to call a few critical stack, which is a security-focused covenanted offering, and Rubicon, which is a ML-based tracking tool. And there is a data profiler, which is more like to ensuring whatever the data you use for your ML-related things to ensure the consistency of the data. So those are the few things, top of whatever we already have for the open source. So I called out a few. We do run some programs like coders. So this program is run across the United States for middle school students who are kind of, we run this program who at least envision themselves as future technologies to see how they can shape up their career. So this is completely run by our associates. So that program is called coders. This is one of the way again we give back to the community. And we do a program called Coda, which is more like non-technology focus associates who want to get into the technology stream. So we run this program for six months, and then we empower with them all the tools and all the things what they need to succeed in the industry. So this is the program. There's a Coda and a lot of our associates come from this program as well. So again, before we dive into this, some detail something. So I am an engineering manager at Capitlone. I have been doing Java-related things and also on the Spark, developing initial days of Java and Spark. And I have written a lot of blogs on our medium and also primarily on big data topics. And I have also spoke in a lot of conferences on the same area. If you're interested, you can hit me in my LinkedIn and Twitter. What do you, Nagesh? Stay here. Thank you. Yes, can you hear me? All right. First of all, thank you so much for showing up here. And I am Nagesh Kumar Vinakota, software engineering manager at Capitlone. And I've been building teams and leading teams to fill full stack applications. And I'm also a tech speaker. And you can follow me on LinkedIn and as well as on Twitter. So what are we going to cover today? So the agenda is like a 10,000-foot view of what we do at Capitlone Rewards. And we will cover a partnership redemption process. So the framework which we're talking about. And as part of that, we'll be covering about our partner specification configurations, which are essential for all the partners. So we call it as 3Ds. So we cover more later in the topic. And we also talk about the data collector. So and also how do we handle the failures in case of handling huge data. And we talk about the restartability and retry as part of these failure scenarios. Then we'll be covering about redemption flows with serverless architecture. And we talk about how do we handle that data publishing part. And how do we ensure that our data is secured and as well as we have processed successfully by enabling the controls and as well as validations. And finally, sending the notifications to the partners. And finally, we end up with question and answers. Okay, before we go into the agenda, how many of you are customers of Capitlone products? Excellent. Ma'am, which one you are using? Is it, which card you are using? Sorry ma'am, what is that? It's a blue card. So venture one. Venture one, okay. And sir, how about you? Which one you are using? I think someone raised up their hand as well. Sir, which one you are using? What's over? Pixel one, excellent. Anyone of you guys from this side? Excellent, like, yeah. Thank you so much. I see like few audience are here like using Capitlone. Thank you so much for that. And like, yeah, Capitlone is using wide variety of products. And Pixel is one thing which gives you 1.5% cash back on all transactions. Likewise, if you're a frequent traveler, you'll be getting like a lot of miles if you're using a VentureX card. Likewise, there are other products, like one of them is like a saver one card. If you are like a frequent person to go to restaurants or dining at an entertainment, you'll be getting like a 3% cash back. So let's say like a small screenplay here, let's say my friend Gokul is interested in like taking a card. Hey Gokul, like, do you have a saver one card? No, I really don't. Okay, so what's your favorite spend? Like, where do you use your spend? No, at least interested in entertainment. I'm not sure about that. I believe like everybody is here, very much interested in entertainment rather than tech talks. Because I agree with me. Excellent. So like, so do you use any Capital One card right now? No, I said already. Okay, so then like, why can't you try for saver one card? You'll get 3% cash back on such things. Maybe I will. Okay, so basically what we want to talk about here is like capital, we came from rewards program, like where a customer will get like rewards in the form of loyalty for the transactions that they made. So what we are going to cover here is, so like we have like a lot of credit card customers like and these customers like one of the customer maybe like maybe Google after he got his saver one card, he went to Starbucks and he swiped his card for a coffee. And well, he will be making like a 3% reward on that. So such transactions will be sent to rewards earn program and we have a series of jobs or processes where we calculate these transactions and we calculate the earn and finally we update our databases so that the customers can use their rewards to perform other action or other redemption processes. Likewise, we're also kind of partnered with other card, other companies like one of them is one of the retail, one of the famous retail is like Walmart. So we are also partnered with that. Likewise, we, Capital One is also partnered with few more companies. So which I cannot disclose right now, but so this partnership redemption process is pretty much like a common generic framework which is across for all the partners. So what we do here is like, we pull all the customer accounts who are like partnered with that further customer for the company and then we will be running this partnership redemption process to issue checks, coupons to the customers. And finally, we also push this data to our data just to ensure that to run internal reports and as well as notifying our customers. So this is at a 10,000 foot view. After that, let's talk about partnership redemption process. What we do here. So each partner has their own specifications. Like one partner will come up and say that, hey, issue a coupon for all the customers who have what qualified with their $25 rewards. Like if a customer is having a $25 rewards balance, they will be like issuing a coupon for those customers. So all such kind of partner specific configurations, we are like coming up with one JSON based format document and we will be configuring those information. And this process, what it does is it reads that partner configuration and reads the data from the database. The data can be scattered across various places which I'll be covering in the next slides. So basically it will pull the data and stays the data by enriching the data or by filtering the data and performing partner specific checks which I called out earlier. Few partners will call say that like, hey, filter out all the customers who are not qualified for this process or else few partners will have an additional checks. They want to send a notification to their customers as well like why they are not qualified for this particular run. In such cases, we perform all these partner specific checks and then finally we have our core set of APIs which does the redeeming the balance and eventually updating our internal databases and finally notifying all our partnership customers. So if I deep dive into this partner specification configuration, we call it as 3Ds here, data source, data select and data sync. So what do we do here as part of data source? As I discussed earlier, the partners data can be scattered across three places. It can be at various places. It could be at tables or it could be in the form of file system or it could be from any other upstream systems. All that information will be configured as part of this partnership specification document and then and also the data select is something where we are saying that like what kind of checks we want to perform as part of these accounts that we are pulling it from the systems. So once the data select and it can also be an enrichment. It is not only just a select kind of a query kind of it, it is more about like enriching partner specific data. And finally, if our framework is, I mean, if the partner is interested in pushing this data to any database, like we'll be pushing that or like if we are interested in pushing to the streams, any kinesis streams like so that other systems can start using it, they can, we can push it to that data sync and as well as file system. So it's purely based on like partners configuration that he's interested in. So before like we are covering this slide, so few years ago, like we have like a series of spark jobs to perform this partner redemption process. That means like each for each partner, we were like having like a series of spark jobs and we are running these machines for all the day. And so our goal is to build a resilient architecture and as well as like a scalable solution which can support all partners and as well as cost effective solution is something which we are considering into it. So for that one, so we came up with this framework which is like reusable and scalable and also we were able to onboard new partners in last time. So which is like huge win for us. So coming back to this process, once we have configured these 3Ds like data source, data enrich and as well as like data sync, the next term would be giving this configuration file to the actual process which is running on a EMR cluster. And this is again a schedule based and as you all know like once a EMR cluster is provisioned and then this partner job will read this configuration for that specific partner and perform all the work it is which is needed for that partner. So let's say once our partner configuration file is ready then let's say if we have like partner A, partner B and partner B and like once the partner configuration is ready the next step would be like giving this file for the dataization part. So as part of that, so we call it as a data collector. So in this case it scans through the configuration file and it says like data source is like hey, you have to pull this information from this data from this Dynamo tables or like from this S3 file or from the upstream systems. Then this EMR spot job which will pull that information and then the next step would be data select where we enrich the data. So as I said earlier like few accounts are qualified for a coupon or few accounts are qualified for a check and few accounts may not and but the customer also want to know why he is not qualified for this run. In such cases, we also send a message so that's where we are doing the enrichment. The question here is like let's say since we are handling with a lot of data if something goes out like if the process got abruptly failed in such cases we really want to take care of not to process any duplicates of it like. So how do we handle that scenarios? In this case for example, once this partner configuration is, he threads the document like A, B, C like three accounts it came up before it pushes that to the data sync it checks the staging table or it checks whether the data is already available in the staging process. For example, in this example, I'm saying like A, B, C out of this like we see that like in the stage table B record is already available. In such cases it will push A and C records to the data sync and once it writes to the data sync it will also update the staging table. Now there might be a question for you guys like hey what happens if the database connections failed for the staging table or else what happens if there is a failure with pushing the data to the data syncs. In such cases we also have handled the retrace logic and in case after the series of retrace if it failed it will also write to a dead letter Q and finally there is another process which will read the data from the dead letter Q and push back the data to the data syncs. So this is how we are handling with respect to the data collector with the restartability and retries and when they call about the restartability again like let's say after like 20 minutes of process like when we are handling huge amount of data after 20 minutes if it failed when it when it restarted it will resume from where it failed. So the next thing here is like we want to cover a board in case like one of the partner is interested in pushing this data to the data sync I mean like Kinesis streams. In such cases we want to cover that scenario. In the Kinesis stream again like we have made completely this as a serverless we don't want to do a batch based processing. In this case and we have a series of lambdas here like let's say lambda A is very partner specific partner specific configuration and where it receives a batch of records from the Kinesis stream and let's say lambda B is like kind of a reusable so where it interacts with like a lot of REST APIs and as well as like maybe interacting with any other source systems. So in this case the record went from the record passed from lambda A processed it and pushed that record to the lambda B and it interacted with REST APIs for writing the data information across our internal systems and there are like lambdas which also do the database interactions. In this case we really want to minimize the number of network database calls that we made so we came up with batch insertions here. So again like lambda C is all about like maybe writing to the database and in this case let's say if any of these messages failed processing by these lambdas so it will also push that message to the dead letter Q and there is another process called retrial lambda. This retrial lambda will process all these it will like reprocess them by pushing them to the corresponding lambdas. For example if you see here this particular process dead letter Q and it also has like this message to be processed on lambda A this message to be processed on lambda B. So this lambda is on a scheduler based and it will like read this dead letter Q and push that message to the corresponding lambda. Well Gokul will cover the data publishing side. Thank you. Thank you Nagesh. So what Nagesh was discussing on the data collect and also the redemption process whatever we have gone through for the particular customer or the customers of the partner. So what happens when it comes to the control and the data validation is something like till this point all the information whatever we have it is only in our internal database. It hasn't actually gone out for the real consumption. The main reason we have some toll gates at this point is we are calling that as a control and the data validation because we want to ensure the accuracy of the data before it really reaches the hands of whom or it needs. So that's what we are calling it as controls and validation. In simple terms the controls is something like the example whatever Nagesh was calling like if a particular account or for the partner if they have defined something like if they haven't spent $28 for that period of redemption process then they're not really eligible to get whatever the means of redemption be it a check or be it a coupon or any of this. So they are not really eligible for that. We know that the partner specific redemption process is going to take care of that. But what control process does is the partner redemption process uses a particular data source to do the redemption but control will go to a alternate data source in order to arrive at the same conclusion and apply the same logic. In this example like customer's account not having $25 it's not eligible for redemption. So that same we do it systematically for all the accounts for the whole partner and for all the partners. So that's what in simple terms we are calling it as controls. And the data validation is something like our DEAs take care of the similar thing but they apply the same logic. So we ensure place doubly all the things before it really goes out for the partner everything is accurate. But as Nages was also discussing not everything goes as per the plan. So we need to have our bases covered. So this process as part of the control or even from the data validation there are ways in which please most not most on a sunny day most of the accounts will get into the case where we are calling it as a thing which makes it to the data lake. Most of the account will get into the data lake and that's when it really goes out to the partner and also to our internal users so that they can generate the business reports and goes out to the partners so that they can really make use of the coupon or cash or check. But there are cases where the controls will find out a lot of followed case even though the redemption has really executed that but still as per our data validation we may not be reconciling. So for those cases we write those things because we don't want to halt the whole process or even for the particular thing. So we go by each and every account and then only just keep the follow-ups as a part of another file then we take care of that as a pack office process and rest of the thing just flows through the data lake and also our partners. So this we have initially done this for one of our partner as Nagesh was reading in one of his slide this was actually previously done as a series of match up specific for the particular partner of those things. So what we would come up with is a specific framework which can be scaled for any partner and use it for any upcoming partners. So we have initially done it for one of the our home improvement chain related partner and then that we have really used it for two or more partners which we have done it in the last couple of years. So that is actually the major win we see with this. That's all we got. What's in your wallet? What's in your wallet? Can you share this? And please show up in level four. We are at S10 and we are like actively doing the hiring as well. So we have our hiring team as well here and you can show up at both S10 and as well as you can also join us today at Zanzibar and we are like also doing a happy hour. Please feel free to join us and learn more about Capital One and we are looking forward your participation. Thank you so much for showing up here. All right, thank you so much.