 Scott Belleware, as Luke embarrassingly pointed out. I really hope now that this presentation lives up to the introduction. So thank you for the extra nerves, Luke. I'll get you back. And I'd like to talk to you today about Avented and Autonomous Services in Ruby. But first let's talk a little bit about why we would want to do such a thing. And I also want to set a little context. When I'm going to be talking about services today, I'm going to be talking about services in the frame of service-oriented architecture, which inevitably means microservices because microservices are a derivative of service-oriented architecture. So we're not going to be talking about service objects, which are those interactor type things that you put in a folder called services under your app folder and your Rails app. We're not talking about operating system processes like you might say that your MySQL process is the database service, or your web server process is the web server service. We're not talking about apps because apps aren't services, and we're not talking about APIs because APIs aren't services. So why services? It's easier. Sure there are benefits that we always hear about like scalability and performance, but ultimately for me it's about productivity. And why is it easier? Because services are focused on a very limited number of things. You would say they're highly cohesive, and they respect all the design principles that sort of guarantee and ensure productivity happens. That's ultimately all about decoupling. Decoupling as much as it is a structural design principle is also a user experience or user interaction design or a cognitive optimization that allows you to think about only the thing that's under your nose and your editor without having to think about secondary side effects or secondary effects. So it's ultimately easier to see mistakes. So we're talking about why we would want to do this. We should really focus here on why we would want to do this in Ruby. And ultimately it's because you have skills in Ruby. You've invested time and energy in mastering the language and the run time in the libraries. You've got a code investment in your organization in building core logic and business logic that's ultimately an asset that you don't necessarily want to throw away. I mean, and this is outside of the fact that sometimes we get to a point in our life like Luke was pointing out that you just sort of top out or bottom out and you just want to leave and you want to do something different and you need to change. You need a lifestyle change. That's another thing entirely and that's fully self-justifying. That's not anything that I'm challenging here. But ultimately if you're a Ruby Dev and a Ruby shop, I think a better question is why you wouldn't build services in Ruby. And there's a career component here. And if you are a, if you have a career in Ruby, you inevitably have a career in Rails. And as you move through your career in Rails from junior developer to senior developer, you end up finding out or you might end up finding out. And here's the good news for everybody who's a beginner. That Rails is its own glass ceiling. You find that everybody who's moving along that line sort of gathers up at the top end of the work and ultimately starts to burn out or starts to move on or starts to abandon the platform. And this effectively starts to have a cascading effect even down the line towards junior developers who can become, you know, demotivated by seeing senior developers moving on or burning out. So I would like to propose that there's an alternative career path where the upper level of achievement is not limited to the monolithic Rails development work that we are so familiar with, where a junior developer can progress to become a senior Rails developer. But then beyond that, there's an entirely other career to be had that doesn't necessarily require you to move on and sort of abandon, not really abandon, but, you know, make that dramatic change unless your lifestyle really wants that, unless you really want to do that. So I mentioned SOA, I mentioned microservices. There's a lot of ambiguity around what this is, what the differences are. Let's talk about that. Martin Feller has a great talk from GoTo Conference in 2014 where he mentions this notion of smart pipes and dumb endpoints. That's SOA, microservices, dumb endpoints, sorry, dumb everything, dumb pipes and smart endpoints. So you can think of smart pipes as things like enterprise service buses, I would consider brokers as well, service orchestration. These are message transports. These are ways of moving around. We're talking about services, we're talking about message-based apps, and we're talking about moving messages around. In the SOA days, we had these really elaborate, fantastical tools where you would put code into the tools, you put intelligence into the tools, then you would deploy the tools to, you would deploy the transports to production and things that we thought hypothetically would be fantastic, like letting senior level people configure and program the tools and let junior level people write the application logic and the application logic can be completely ignorant of the fact that there's messaging happening. That never really works out at all, and it didn't really work out well, and also these tools had state in them, they had databases, they had configuration and moving that stuff around and getting into production and updating it, it wasn't as good as we thought it would be, and when we were debugging those things in production, well, there was a lot of debugging that had to happen in state. If anybody ever used RabbitMQ at high-scale and high performance, you might have experienced failures in RabbitMQ's underlying message store, I've seen this in high-scale, high-perf scenarios. It's no fun when you can't actually just figure that out in code, because the intelligence is not in a form that is debuggable. So, messaging, what's a message? Message is just basically an object that has some attributes that can be serializable and serialized and get sent out on a transport. What's a transport? Well, let's say we've got two services, two nodes, two things, and a message, an instruction is going to be sent from one service to the other. Basically, we instantiate a message, give it some data, and send it over the wire or over the transport to the service, the other service processes it and then that message is removed. It's gone, it's been processed. Let's see what happens when we do this in message brokers. Same thing, service one, some, you know, service on the left sends a message to service on the right. Service on the right is going to receive that message, process it, that message has been processed so it's then removed. That's not really what's happening. So, in detail, the message is being sent across the wire, it's processed, then the other service sends a message called an act to the message transport. The message transport gets that message and how it's interpreted is, this message has been processed, we don't need to see this anymore, remove that message. Challenge is, this is running on a network. And because it's running on a network, once it's processed and the act message is sent, there's absolutely no guarantee that that message will ever be received. Why? Because it runs on electricity. It runs on a network, it has a network, it has computers, it has electricity, there's no guarantee that a message will ever arrive. So, what happens when that message, what happens to, sorry, just doing this to reinforce it? So, what happens to that message when the act is not received? Never leaves the queue. It gets re-sent and effectively, unless if you're not careful with it, reprocessed. So, what if that message over there is, let's say the message that says, process your direct deposit for your salary. If you're the bank, not terribly great, let's say everybody here works in the same company, everybody makes X thousands of dollars on their paycheck, and all those messages get processed twice. The bank is going to give everybody in the room a double paycheck. Yeah, it works that great for you. All right. Event streams. I like this quote. This was from one of the authors of a database, a special purpose database for message storage called event store. And it says, we were talking about the differences early on in my career about the difference between queues and streams. And he said, what is a queue but a degenerate form of a stream, which suggests that streams can do everything that queues can do, queues can't do everything that streams can do. So, in event streams, we're effectively having the services write messages to basically to to an endless log. And the other service is got a reader or subscriber, whatever you want to call it, a pointer. And it picks up each message in that stream, merely by incrementing the number the message number that it's reading. The big difference here is messages aren't moving anywhere. They're not being transported. The only thing that's changing is an incrementer or an offset where you read the messages. So the message stay the same. They never get lost. They never get act. It's a lot more simple of an implementation. But it's not really like all one service writes all the messages and the other service reads all the message. This is happening all simultaneously. So the other side hopefully didn't didn't misrepresent. This is what an event stream would look like if you model it in a relational database. But there are special purpose tools that store are specifically meant to do event stream storage, like I mentioned, the event store database, eventstore.org. This is done for Postgres. You already know the URL for that. Pub's up. Publish and subscribe. So the same model, the thing on the left is a publisher, the thing on the right is a subscriber writing a message to a stream or to a queue would be considered publishing and reading that message would be considered subscribing. Commands and events. Right. Let's say we do have this bank or money application, this message that we're going to send and we've got a service that deals with that deals with the with the accounts with the bank accounts or GL accounts. A command is a thing that we send to the service that then gets processed. The result of processing a command is invariably usually an event. And this common pattern is really common. It's foundational. Passing a command, process the command, the result of doing that is going to be an event. Event records the effects of processing the command. So here's the same example with a little bit of data, a withdraw command has an account ID 123 has an amount has a time stamp. The withdraw event has all those things except we're also adding another timestamp which documents when the thing was processed. Command on the left, vent on the right, we process these things. What is happening when we're processing these things? Well, in this case, effectively, we're copying one of the things that happens is we're copying data from the command to the event and recording that happened. This is a little bit of an oversimplification of what happens, but this is a really common case. In most cases, you're simply not simply in most cases, you are also, you're going to at least copy data from the command to the event and then record it. And then add a timestamp to say when it was processed. So what's really happening when we're processing these things? This stuff. Validate the command, as you would, let's say, an HTTP form. Retrieve the account entity. I'm using the word entity. I'm going to reuse this a lot. If you were Rails developer, you might know the word model or model object. Model object is an entity, is a specialized form of entity. I'm going to use the more generalized term. So retrieve it the same way you would in, let's say, a Rails web controller, where the first thing you do is a find on something and then change its attributes and then save it back. So retrieve the account entity, do some business logic. We decide whether we want to reject or accept the command, which means, in this case, if you want to withdraw $100 and you have $10, then you would reject the command with insufficient funds. And then we construct resulting events. By the way, the insufficient funds is an event and then write the event. Here's what that logic would look like. Actually, here's a better example of what that logic would look like. It retrieves the entity, as I said. Here's the line it does that. We use this thing called a store. By the way, this code is based on a toolkit called eventide, eventideproject.org. And it's a toolkit for building evented autonomous services in Ruby. So there's an object called a store, which is kind of like an active record thing. An active record, you do a find. The syntax here is fetch. And that gives you an entity. Then we decide whether to accept or reject the command by doing the business logic here on the account object. That's why we retrieve the account object. And then we decide whether or not we're going to reject it, which is the side effect of that is simply recording an event or accept it and the side effect of that is simply recording an event or constructing an event. I should say constructing event and down at the bottom we write it. Event sourcing. If you've got PubSub and you would want to have PubSub because it's a desirable pattern when you're doing distributed systems and services and autonomous services, then you can have event sourcing. What's event sourcing? In an event stream, you will have, let's say, back to this account example, you have a bunch of different events, deposit, withdraw, deposit, withdraw. The event stream's account dash one, two, three, that's really sort of not really entirely pertinent. But this is the kind of thing that you would see underlying or underpinning a service. Now you have an entity. And I said these are like active record models. So hopefully this is somewhat similar or familiar to you. It's an object with attributes on it. And the attributes have values. In this case, it's an ID, a customer ID, the current balance, the time that the account was open and the timestamp of the opening, timestamp that was close. Here's what that looks like. It's an object. I mean, there's not a lot to say about this. If you've worked in Rails, this should be a little bit similar. If you've done any OO stuff or DDD style stuff, this should be relatively familiar. Pretty simple, dead, simple object. It doesn't do anything, doesn't talk to the database, doesn't talk to networks, doesn't have a mailer, doesn't have validations. All those things are separate concerns, so they're not in this object. Event sourcing is done by projection. So here's the event stream with actual data now documented. We've got an open event, then there's a deposit, then a withdrawal, then a withdrawal, then a close. Of course, an actual bank account can, you know, my parents opened a bank account for me when I was eight years old. They don't really just last four events. They can last a lifetime. You can have these things for multiple lifetimes. So this is just sort of a hack-need example where the account only survives for five events, but ultimately this could be infinite. In projection, we start with an empty or new, we knew up an account object, all the attributes are nil, and then we sequentially read the event stream. The open event has interesting piece of data on it, interesting pieces of data on it that we will apply to the account object. So when we read that data, you can see up here in the animation that these attributes are changing. So the open event provides the account ID, provides the customer ID, and it provides the opening time. A deposited event is basically going to affect the balance. So here we started with a balance of zero, we're depositing $11, balance is 11. If you remove one of those with a withdrawal, yeah, if you remove $10 with a withdrawal, that balance of 11 goes to one, hopefully, you know, that's self-evident. The next withdrawal takes out the last dollar and the last one is a closed, which effectively just sets the closed timestamp. That's event sourcing. This is a slide that shouldn't be here. No, I'm kidding. All right. I just blew it. This is the line of code where the event sourcing is happening, where the event projection is happening. It's transparent. So even though this might seem really onerous and difficult and why the hell would I do that? Why would I write code to do that? The code really in your application logic effectively looks like this because this can be put behind the scenes. But this is what a projection looks like and it looks like a handler. And effectively, let's just take a look at the opened, when we're applying the opened event to the account, we're effectively stripping, so here's the message object, the opened event. We're stripping values from that event and putting it on the account object. So account ID is the open message account ID. Customer ID is the open customer ID. The time is etc. When we get into stuff like deposit and withdrawal, there's a little bit of business logic, so we're actually going to do a deposit and do it with y'all. And because we're using fairly good domain modeling and domain programming practices, that logic is on the account object because I already showed you want to show the entity. So we're basically calling deposit with the amount, and that amount is on the event. This is a great quote from my partner. Event sourcing is the path of least resistance to microservices. Let's talk about autonomous services. So we did the evented part. Let's talk about the autonomous part and autonomy. Autonomy is one of the four main tenants of service-oriented architecture. We're not going to go into all what all the other ones are, but ostensibly if you don't have a service autonomy, you don't have services. You effectively have applications. Nothing wrong with that, but if we're going to call something a service, we should call it a service, and should we know what that means because there's extraordinary benefits to doing things this way once you get used to the style of programming. And what autonomy means is that you can take any service offline at any moment in time, and none of the other services fail. When would that happen? Well, if you ever upgrade the code on one of your services or deploy new code, that service is going to come offline, even if you're only doing it momentarily. So the ability to take a service offline, now the word for that is to make it unavailable. To make a service unavailable, you should not have cascading failures to the rest of your infrastructure. Unfortunately, when we start out as web developers and we try to do this microservice thing, there's a preponderance of cases where we as web developers will sort of go in with our web development biases and we'll say we're doing services and what we're effectively doing is a system of distributed web apps that are all tightly coupled rather than being loosely coupled because they don't have autonomy. And when you take one of these things offline, all of the other things go offline as well. And then you get into a lot of really sort of fantastical countermeasures like circuit breaker and I'm not going to get into what the details are. Those are valuable patterns, but they're not really things that you need to deal with this problem because this problem really shouldn't exist to begin with. So the number one rule of service design. Services do not have get methods. You can't put an HTTP get API on a service and allow that service to have autonomy and a service without autonomy is not a service, it's an app. This is the most common thing that happens when web devs go into service architecture. You got to look at it this way. Well, you don't gotta, but it's helpful to look at it this way. Services do things. They don't represent things. So the behavior, they're not the data. When we use services as data things like go and get the data from that service and then do something with it. We already have a name for the thing for a kind of thing. We have an architectural archetype name for a thing that you can go and get data from. It's a database. Yeah, yeah, sorry. It's a database that the fact that you put an HTTP API around a database architecturally means it's still a database. It's just got an HTTP API. If you use something like Oracle and or SQL server not saying that you would or you should, but if you have to, you'll know that these databases actually come out of the box with the ability to put HTTP interfaces on the database without creating a Rails app. There's more insidious things that happen when we do this. Here's an example of a product table or product model. Hopefully this is familiar to you. It's maybe not realistic, but hopefully it's familiar. If you were building like a little e-commerce, not a little, maybe even a big e-commerce app, but if you're building some kind of e-commerce app and you had a product, a notion of product, a product model, hopefully this example makes sense to you. It's not alien. It has a name, a description, a price, a quantity in stock, maybe a URL to an Amazon S3 bucket where the picture of the image is. This is about right. Y'all, anybody disagree with this, except on architectural terms? This is familiar? Great. Here's the issue. This is not really the data of one application, let alone one team, let alone one department, let alone one company. It's certainly not the data of one database table. This is a monolith. Now, we call the Rails app a monolith, but this is the monolith. Monoliths are databases, are database tables, are database schemas. They're the design of the row. When all of the information for the product or the user or the thing or the other thing or the this or the that or the yet or the yet up, when all that stuff is in one row, that's a monolith. That's what makes it difficult to get from a service architecture, from a monolithic architecture to a service architecture. The monolith starts at the database. When you put an application on top, when you put an application on top of monolithic rows, you get a monolithic application. Let's deal with some popular myths while we're on the subject. Myth number one, you need to switch languages to build services, go, Elixir, Scala, Java, et cetera, et cetera, et cetera. Language has nothing to do with this architecture. Certainly, you don't want to have to go and build a web app in Bash, but Ruby's a good language. No, no, no. I mean, if you do, I mean, all the power to you. I've got some scripts I would like to have someone take a look at that I don't understand. So if you've got a language you can build a web app and you've got a language you can build services in. Sidebar, the moment you adopt a new programming language is the moment that you are the worst with that programming language. Being at that level of skill in a programming language and adopting one of the most difficult to get your head around architectures that you are also a beginner at basically turns the success probability of your project into a coin toss. Unless you've already got prior experience, which means you're not actually a beginner. But being a beginner, both language and architecture, that's the highest risk project anyone can take. That's why managers look at the big rewrite and say no. Because the big rewrite usually comes with an implicit change of language or implicit change of architecture. So you should try to change one of those at a time and not both. Myth number two, you should use containers, Kubernetes, Dockers, et cetera for services. Services and containers have nothing to do with each other whatsoever. You can deploy your web app into a container or you can deploy it onto native. You can deploy services into a container. You can deploy it to native. It doesn't matter. This is a DevOps concern. This is entirely an operations concern. The reason why we have a lot of the hype or the story that is being repeated that services and containers belong together is that we create services that are in fact distributed monoliths, i.e. we put gets on them. So we have tightly coupled services rather than autonomous services. And you get a scenario where like, well, look, you know, I don't have to start all of these services all at once just to develop and test. So I'm going to put them all in containers. That's a perfectly reasonable statement. The problem with it is if you have that situation, there is another mistake that needs solving. And that's the mistake of not having autonomy. Once you have autonomy, you can work on one service at a time without any considerations of any of the other services you have. When I work on services, I don't have the entire universe of the rest of the services in your organization. And I don't test through them. If I did, those would be applications, not services. Which brings us to it's all about APIs. Services are about APIs. Yes, a service has an API. But when we say API is web developers, we're inevitably thinking HTTP API. The services that I talked about, the autonomous services, have APIs. They have interfaces. Can anybody sort of suggest at this point what those interfaces might be? Yeah, they're the messaging. Yeah, the events and the commands. That's an API. It's just not a synchronous API. And it's not a tightly coupled API. The other one is that message brokers are the way you do microservices. So you would inevitably use RabbitMQ or Amazon SQS. These are all brokers. Kafka is another story. These are all brokers. They have that whole thing with the act. You've got to be really, really careful with that. It shouldn't be presumed that you're going to use a message broker or a smart pipe to do services. And myth number five, a monolith is an evolutionary step on the way to microservices. This couldn't be further from the truth. Microservices and monoliths live at opposite ends of a continuing spectrum that are often not reconcilable. The reason we end up with monolith projects trying to become microservices and ending up somewhere in the middle is that there is no real, natural path from one to the other. What ends up happening typically is web developers will create a distributed monolith out of services that have get methods. And those services are not really services. They're just sort of applications that have entities in them. So you end up with the user service, the auth service, the product service, and all the services look like the models you used to have in your Rails app. And services aren't models. Models are representation. Services are functions. Services are doing it. That doesn't mean they're just Lambda functions, but they are doers of things. So some quick clarifications called the service. To really start thinking about these as accounts, a component, I should say, an account component has a service. The service is just the infrastructure wrap around it that allows it to interact with the computer. So I could put multiple components into the account service. In fact, at that point, I might rename the service. In fact, at that point, the service can be named just about anything. Testing. I don't want to belabor this point. But testing. Input, process, output. A command, a message goes in. You run a handler on that message. An event goes out. So you create one of these. You actuate or activate or run this, and then you validate the attributes. This is the least amount of testing that you can get away with. You really should be testing handlers, projections, entity logic. And all of this stuff can be tested without actually using a message transport. All of this stuff can be done entirely with in-memory substitutes of all of the infrastructure. And the other thing you need to test that we haven't really talked about is idempotence and concurrency. There's not really enough time to get into that. But if you try to build this stuff without taking care of it, and if you look at the examples online, you will see that the handler logic is quite a little bit more elaborate because we have to take care of idempotence and concurrency. This is what the architecture in the end ends up looking like. I've only talked about the services, which is this part over here. The rest of it is applications data aggregation and view data. And this is an architecture that is commonly called CQRS, where over here on the left, you have services receiving commands and emitting events. And then those events are effectively read and turned into database tables or database views or whatever you want. And that data is the stuff that gets painted onto the UIs. That's about all I have. If you want any more information about the project, which is entirely written in Ruby, take a look there. I'll add the links to these example apps are there. If you want to follow me on Twitter, I am rather combative. So if you're not into that kind of thing, avoid it. And that's all I have. Thank you.