 Good afternoon everyone. So I work at Honest B and my talk flows in very well with what Florian just discussed about monolithic applications And oftentimes what you find is that as your application size grows bigger and the business requirements keep flowing in Certain parts of the application You know, they just get bulky and it becomes harder to change them for various reasons I mean if you if you don't have a sophisticated system like Shopify does and you know Sometimes you have to wait longer for bills to run deployments become slower Tests need to be you know, you have to have a powerful set of test suite And if you don't if you don't feel confident about certain integration tests, you know Surely you have to have put in QA efforts and so on But obviously splitting up applications is not Something it comes with additional responsibility and there are certain Cons, you know or rather certain additional like headaches that it brings along So I don't want to get into whether you should or you shouldn't split it split it up because I think that's another topic All together, but should you choose to do it? What are some of the good ways or what are some of the ways to think about it, especially from a development point of view? Not thinking about the deployment challenges involved in this process so I'd first like to show you this diagram and Let's say you decide to split services into you know different You take a big service and you start taking out some of the you know some of the components of the service out There's various ways for these services to actually talk to each other and one of the common ways is Http now one of the issues with services talking via API's is that Every single service is aware of the concern of the other service. So let me give an example let's say you have a service that creates users and You also want to do this additional notification, you know logic where every time a user is created Maybe you want to send a notification for verification of the user or send out welcome emails and so on and Notification itself is something that can be that is useful across your system not just for users So maybe you think about hey, maybe I can separate out notifications into a separate service Now if you were making your two services contact each other via API's You know user creation service now has a contract that it needs to follow and it needs to know that at some point when the user is Created successfully it has to send out a you know an API call to another service on the right you have a slightly different approach to this and You may choose to use one or the other. It's not a complete black or white situation But the benefit of the second approach for at least the example that I just spoke about is that your user creation service only cares about creating a user and Once it's done with that it pushes that event out into a message pass and the services that actually care about Anything to do with user creation decide to then pick it up pick up that event and do certain things like sending out notifications And so on so in a way you're reducing the coupling between your services And you're also separating the concerns of different services quite clearly when you use such a system So fundamentally it's this Philosophy of fire and forget where as a user creation service I fire a message and then I don't really care what happens and then another service picks it up and it processes it and Once it's done processing it maybe prints an acknowledgement back And you know if a service cares enough about it they pick it up and they continue with it And that's how the flow continues Now there is various tools to do it and once again I wouldn't want to get into the debate because Everybody's use cases and situations may demand different tools But let's say you choose to do it using Kafka, which is actually a highly scalable Distributed tool it provides a certain replication factor. So, you know, it's highly reliable and so on so been used by a lot of organizations so for those of you who may not have worked with that some of the terms just before we get into a More deeper example is a concept of a producer. So you have producers that are constantly You know just publishing messages to the bus and they're doing it in a particular topic Which is basically, you know, what what I if it's a user creation Maybe you have a topic called user and so on right and these topics are then divided into partitions And the benefit of splitting things into partition is that it can be it can exist across multiple machines So it makes it highly, you know scalable and then you can also have your consumers subscribe to Certain topics and read it simultaneously off a particular partition So you can have multiple consumer processes that are running and that are simultaneously reading it So it also gives you this whole notion of parallel processing of messages. So that's what makes Kafka highly superior and a Really good choice as a message pass. So some of the key terms are producers not going too much into detail of you Know all the other stuff that happens in the background But you have your producer consumer and your topic and so let's talk about the simple use case that I spoke about in the beginning Which is a user creation and maybe just sending something as simple as like a welcome message to the user saying hey Thanks for signing up so the first thing you want is a producer and The producer is the one what we found out in our rails application context is that oftentimes the events that you Want to send out are linked to Database changes so it could be a insertion to your database So it could be an update to your database and oftentimes these are the events that you want to push to the message bus There might be others but these actually compose majority of the events that we wanted different applications to respond to and so one way to do that is To have code to send out messages spread across your application and That definitely brings around a you know a bit of duplication So then fine, you know you bring it into callbacks in the models Some of things you know when you when you allow different developers to just add logic to send messages to Kafka based on certain events is that To an extent you want to have certain contracts followed even when you publish things to Kafka So for example metadata, right? Maybe you want to have a timestamp included in every single one of your messages So where I'm driving at is that it does make sense to Centralize the logic of sending the message itself and adding some of the metadata attributes the contents and the topic of the message can be Configurable but some of the other factors around it remains like us, you know, it's a similar logic across the base. So What does it look like in practice? So basically this is what we ended up doing we created a concern that allowed Let me just show it to you what it looks like so it just basically is something that can be included inside your model and You just had a simple published statement with arguments that are fundamentally erased because you do have situations where you want to publish Multiple times from within the same model. So for example when the user's creation I want to send a message to a particular topic for user creation and I want to use an existing serializer that Decides what the format of the messages and when the user's updated, maybe I want to only send a message conditionally if the phone number of the user's updated and and at that point I want to have a different like a message blob, which is you know coded inside based on the new attributes like the phone number and so on right so you can have an array of messages that get sent out for each of your for each of your models and That's how you have your first, you know a simple producer where Every single creation or update can be then published to the message bus now as and when you're publishing messages at some point You do want to pick up those messages and process them and do something with them And that's where one of the libraries that we discovered is caravka Which makes it really really easy to set up a very simple and clean structure across your application where you basically use You you create a central configuration and you register a controller to a topic So you can create applications which are actually having consumers consuming from multiple topics or multi-topic Applications in a way and once you define this mapping so you can see that it's actually just quite simple configuration You can define your controller code and I mean it's advisable to keep your controller code Lean so yeah So what I've done here in this case is that I've just created a separate service to send out SMS in this case and I'm just picking up the The the phone number of the user as well as the name of the user from the message that was sent to the message bus And that's it and that's where your consumer logic is so I can show you a running example I haven't advised not to do demos and short talks because you know whatever Has to go wrong will go wrong But I'm just gonna do it anyway because I just want to show that it's actually really easy at least from a development point of You and you know the barriers to do it is really little just because we have a really good ecosystem at least in Ruby to do this so So this is my code that I was just talking about and this is the this is the application that basically Has the you know, it's the same code that you just saw on the slides So a little bit redundant and then this is where your controller is which is exactly the same logic and When I go into my terminal oops All right, so what I have here is I have my real server running on one end And then I also on the right hand side in this particular window I have my caravka server running so caravka just a bit of background is that it takes the messages in the message bus and it pushes it into Sidekick cues and then you have sidekick workers that are basically picking those messages up and Sending it to your controller and the controller then processes the message so the right-hand side is the Caravka server and the left one is the caravka worker. There's a few errors and that's all my local environment configuration It's not entirely normal, but I haven't bothered fixing it And then this is my sign-up form right so this is simple sign-up form And what I'm gonna do right now is I'm just gonna create my Just add my phone number in and then add my honest be email and then just you know a password and Here's my phone So I'm gonna sign up and ideally I should get a message if all goes well so there should be some logs which basically says that it's done processing and Just about any time now There so I got a message just now which basically says thank you for signing up so What I'm trying to say is that from a development point of view it's easy there are deployment challenges But for now let's just leave that to the DevOps to take care of and let's just focus in developing it. Thank you everyone