 Saya sangat gembira berada di sini. Ini sebenarnya pertama kali saya berada di sini. Jadi saya sangat gembira berada di sini. Jadi apabila saya sangat gembira, saya akan bercakap dengan dengan dengan cepat. Jadi anda tahu, saya akan berhati-hati dan saya akan melakukannya. Jadi, saya rasa anda akan melihat web-side dan saya fikir titik itu sangat lama. Jadi kami akan menghubungi Komare dan kami memutuskan bahawa kami ingin sebuah titik yang lebih kecil di sini. Jadi, today's topic is about managing complexity of DSLs and a little bit of matter. So, a little bit more about myself from Singapore. Just a short talk away so if you have some time you can pop by. But otherwise, if not, you can go and catch what's that show. Crazy rich Asians. You will see Singapore there. Except that disclaimer most of us are not rich. A little bit crazy maybe. So, I'm an engineer at Ascender. We're a loyalty technology company. We'll talk a little bit more about that later for some context. I work with a bunch of very talented engineers. In fact, some of them are here today. And today's topic is about the collective experiences that we've had as a team when dealing with complexity. Lastly, I enjoy reading. I do some running mostly so that I don't get fat. And I can eat more Pad Thai. And I do love movies. Wish I would think I will need your list of movies later. So, speaking of movies, has anyone seen John Wick? Cool. So, I was watching that on a plane over here. And I realized that there's John Wick 3 recently. And I'm not sure if you noticed it. When he's back for the third time, he's basically still trying not to get killed. I've watched it once, I've watched it twice, and I've watched the third one. And it's always the same thing. So, sorry, technical issue. So, yes, that brings us to today's topic. Have you had projects that are basically coming in different forms? Just like John Wick fighting the mobsters. You've got the Japanese, you've got the Russians, and so on. But he's always fighting someone and trying to survive. So, our projects are something like that as well. It's a different context, but yet the plot is almost the same. So, this is where we talk about how DSLs help to solve some of the problems that we had and how we approached it. So, to guide us a little bit more about this, we need some context. So, unfortunately, AMACS is not one of our sponsors, but here's a little bit of an example. So, I want to fly to Thailand to come and meet you guys. So, I want to redeem some points so that I can buy myself a ticket on Thai Airways. So, I go to AMACS, I redeem my AMACS points for Thai Airways points. And from there, I get my ticket. So, that is kind of the context of today's project that we'll be looking at. Our product is essentially a points exchange. And you multiply that by 20X. So, again, it's not advertising, but here's a little bit of an example. So, you could be getting points from Kung Thai here in Thailand, and then you want to exchange and maybe fly via Emirates. So, this part is brought to you by Emirates as well. So, multiply this by 20X. And it starts to introduce some scale-related issues when it comes to development and maintenance, when you're working in a sizable team of developers. So, every little integration is different in its own way. They all have got their own little habits. So, like, see when John Wick, he tries and fights all the way up to a big guy, they first come to him with their fist. It doesn't work, they use knives. It doesn't work, they start throwing axes. So, it's kind of like that with integrations as well. Everyone is a little bit special. And add on that, you know, at a project level, you've got new connections, migrations, enhancements. And, you know, as the business always says, we want this in yesterday already. So, we started to face issues on, like, you know, how we could keep up with demand at scale. So, here's a little bit of an example on how some of our content may look like. So, for example, this is a simple CSV file. We've reduced it to the essential pieces of information. So, for example, you're exchanging points and you'll be submitting your membership ID for a loyalty program. So, here's a little bit of an example over here. Of course, using this as a case, then we've got many other types of data. So, you can have it in TSV, fixed-wave files, spreadsheets, whatever, everything. And, of course, we've got APIs because everybody loves RESTful interfaces, right? So, I want to show you a little bit more of an over version over here because you've got this very weird JSON request body over here. So, here's a use case from one of our partners and it also gives us an idea of some of the issues that we face with. So, for example, if this partner they can't deal with, or rather their systems can't see, for example, the HTTP status codes, nor the haters that you typically expect in a request. So, everything goes into the body. So, here's an example of some of the varied situations that we see. So, other variants include, for example, inconsistent response formats. Sometimes you get a JSON and then a different HKs could be returning you in plain text and so on. And then, of course, you've got a bunch of feature-related enhancements as part of the project. They might say, I need you to implement like a client credentials grant to authorise you making the request and so on. And then, you've got pre-processors and post-processing. So, here's a sample of how the code looks like. I know it's quite a bit of a mouthful so I'll take you through it one by one. It really is just your typical Ruby class that helps to execute the sending of a CSV file. So, if you've got a bunch of conflicts, for example, like your file name, partner name, and so on. And then, you create a file and then upload it and so on. So, we actually had something functional here. And it's decent. It works up front when we first launched this project. However, when you multiply that by 100x, you start to see, you know, hey, this is maybe not so sustainable. In yesterday's talk with team, we talked about delegating responsibility, right? And yet, this class seems to be doing almost everything. So, we don't like that. That doesn't spark joy. So, this was our individual developers, and now it's good. This works. Nothing to complain about. Then, when the other developers look at this, like, hey, we need some Marie Kondo in the house. So, a year later, when you have to go and update the code, like, goodness, what's going on over here? You're like, you know, as you scale and as time goes by, as the code deprecates and the partner says, you know, hey, we've just updated in the phase yesterday. Now, everything is failing, so you've got to make it work. I want everything to be okay by tonight. So, then you go back to the code and you're like, oh, shit, what's going on over here? So, obviously, it no longer sparks joy. So, for us, the goal was this. How do you standardise the approach of handling integrations? Can we focus on only what's custom, what's special and extract away what's common? So, for example, when you create a file, if I expand it by 100x200x creating a file, it's always the same. Maybe what's different is maybe, let's say, the delimiter. But everything else actually stays the same. It doesn't have to be appearing in that same file. So, we want to take this and apply it in all its different variations that we could possibly work with and make sure that we come up with a partner file that only has customisations, almost like a configuration file of sorts. So, like John Wick, when Betty No. 199 or integration No. 199 comes along, right? We want as a team to be able to respond as quickly and as flexibly as him. You notice, whoever comes at him, he just shoots them in the head. It's always the same thing. So, we want to do the same as well. So, for that, actually, consistency is key for us so that when you're consistent, you're quick, you're flexible, like John Wick, hopefully, and then that helps you to simplify the domain complexity of the problem. So, just very quickly, we identified three main areas that needed addressing so that we get to a very consistent state. One was the data. So, the data in this example over here is essentially two different formats, but they all achieve the same goal. It's just that the way you send it, the way you hit the guy, it's the same. So, of course, this also contains the business logic of the information that you're sending. And then you've got the delivery mechanism. So, like John Wick, in order to get to the final boss, he has to go through many stages. So, this is an example of an interesting partner that uses a layered architecture with an entire organization. So, instead of talking to the final application, you have to go through a bunch of applications, actually. So, the first one gives you the response in plain text. The second one gives you a response that's not in XML. It could be in JSON, sometimes even plain text as well. And then only then you get to your target system for which you want to fulfill your points. And that's the only one that responds in XML. So, lastly, we've got customizations. So, as the business says, you need to make everything okay. So, this was also one of responsibilities that our configuration had to handle. You needed to be able to jump in quickly to modify as needed. So, this includes special things like, oh, you need to encrypt the data. You need to be able to decrypt the data. Authorizations. You need to deal with time zones. So, this really was our goal to standardize what's common. Make every integration a configuration if you can. And each class only focuses on customization. Because this would make sure when we are refactoring our code over here, hopefully we don't have to go to the moon. We want to make it a bit simpler. So, we've actually got four steps. But for today's presentation, we just focus on two. One is preparation and the other one is delivering the data. So, here's a sample of how our DSL look like. And then we'll dive down a little to take a look at the individual parts that enable it. So, really the big idea over here was to firstly consolidate and simplify the high-level configuration. So, remember the previous file had a lot of the specific conflicts for each major type of delivery and preparation of the data. Over here, we want to put it if possible into a single line. So, we had a builder class. Then, for the rest of the file, you really just focus on what's custom. So, only if it's custom, we'll put it in here in the file, if we can. So, you notice over here, we've got the formatting configuration. But for example, we didn't put any configuration that was delivery-related. And lastly, we've got the configuration of the data, the content itself. So, this one, we will not touch too much about it because this part would be pretty straight for its business logic just like any other application that we've done. So, the first part, your high-level config. So, this is enabled for a builder class that we include directly in the partner class. And this is how the builder looks like. So, essentially, we extend a module because we wanted the dynamism to be able to specify what are the individual component classes that we want to include. So, this is where the dynamism comes into play. So, over here, by specifying the config, you say that I want a CSV or if I want to make a SOAP request. And then this is where we compose the modules in. So, here's an example of what we compose. It's an example of a CSV file. And that's really just two main things over here. So, the first section, which is kind of the meat of the logic, it's us trying to be able to define methods on a fly and assign a certain value to it. So, for example, I want to specify in a single line. Hey, my column separator is let's say a curly brace. And then this is where it happens. So, you want to be able to define a method. So, we'll talk a little bit more about how this is enabled. But essentially, here is where you specify on how you enable your CSV configs to be applied. Then the next part, preparation, that will be your execution of your business logic. So, this part is fairly standard. Once the first part is already done where you're configuring your values, then the next part is just executing the logic. So, over here, also there's two parts. So, firstly you've got your define config. So, remember just now we had configure your column separator or row separator. So, this is where it all happens. So, you want to make sure that we only customize what's needed. Otherwise, you want to be able to default it. So, for example, over here, the config method receives the config name, which is essentially when you define a method that will be your method name. Default value and a block if you want to supply some slightly more complicated logic. So, that's your first part. Your second section is where within the partner class, you want to customize the logic. So, this is where you define the method that allows you to accept the customized value. So, I'm running out of time. For more information, you can actually check out the Jam Simple Configuration that was written by our VP. This covers this logic that you see as well. So, all in all collectively, this allows us to build a pretty concise DSL that allows us to represent a partner. It's extremely efficient because considering that your domain complexity involves a lot of very small methods and values with very simple logic, this really makes it easy for the individual developer to work on changes or construct new integrations. So, actually relative to our prior classes, we managed to reduce the lines of code by 50% or more. And the classes are just very much more economical. So, in summary, here are the benefits. So, development is really fast. Testing has been simplified because we've extracted away what's common. You really just focus on testing and customizations. And it's really easy for newcomers to join us and work on a very core part of the system. Of course, on the flip side, because there's always a pricey of the pay, right? The underlying code is not so easy for us to modify, especially for new engineers. And testing can be a bit more difficult because this is essentially a building block of our entire construct. Although, at the same time, a nice side effect with scale is that when you've got complexity at scale, right? If we all write enough specs, it generally helps to cover all the possible cases that you could handle. So, even though you're modifying a particular building block, it helps a lot when you have a very good test coverage across all partners. And yes, that's it. I hope you enjoy the talk. Hopefully, we see John Wick 4. But this is us when we get a new partnership. I'm thinking I'm back. Thank you.