 Hello, everyone. Can you all hear me in the back as well? Excellent. You're gonna have a really boring time if you can't My name is Sebastian I Live in Australia. I'm originally from Sweden So if you I like to tell people where I'm from before I start talking Otherwise, they spend half the presentation trying to figure out where my accent is from I Work for culture amp. I'm the VP of engineering there and until About eight months ago. I was working for another company in Australia. They're called Envato Where is which is where I started first doing event sourcing about four years ago four or five years ago? so Today our agenda is we'll talk a little bit about event sourcing We'll talk about event sourcing using another pattern called CQRS We'll mix in a little bit of the major vendor sign into that And then before we kind of all go home I want to tell you a little bit more about kind of like the upside and the downside of these patterns as well I don't want you to leave here thinking that this is a silver bullet that will solve all of your problems There are some downsides as well And I want to give you as balanced the view of this as humanly possible So let's get stuck in I'm gonna go through my slides quite quickly So if you're trying to take a photo of a slide that I've just skipped past Just let me know and I'm happy to go back for you But let's start with event sourcing. What is event sourcing? Well, many of the systems including those that you might work with today will be logging events of some kind It might be you know your logging requests over into a log file over here But it's really important to know that event logging is not the same as event sourcing They're in fact two very different things Most systems today I found tend to be built built around some kind of database that you use to store current state and an event log if you're doing event logging you The event log would kind of sit to the side Event sourcing kind of flips that around a little bit by forcing all of your Current state to be sourced from your events. So so hence the word event sourcing In an in event sourcing, it's the events themselves that are your source of truth for your system Before I go into What event sourcing is I I I'd be amiss if I don't call out this man. His name is Greg Young He's a bit of a pioneer in the space. He is the pioneer in this space He is the person who coined the word or the term event sourcing and also the person who coined the term CQRS But it's important to know that he didn't come up with it in the first place He didn't invent event sourcing because event sourcing is actually not new at all In fact event sourcing is at least 500 years old Greg says that he's been able to Track event sourcing all the way back to ancient Mesopotamia about 4,000 years ago I'm not sure that I buy that but I do buy I do know for a fact that it's at least 500 years old and the reason For why I can say that is because 500 years ago was when the double entry accounting system was first invented The reality is most mature industries that we work in That exists today are already inherently event sourced The finance world the accounting world insurance medical legal all of these industries are event sourced Take for instance your bank account. There's some a lot of people here who works for a bank here I So quite a few quite a few people I think I Have never worked for a bank in my entire life and I'm going to use a lot of bank examples So I'm very likely going to make a fool out of myself, but that's all right. I Presume you all have a bank account, right? It's where your salary gets paid into it's where you pay rent from so on and so forth Do you think that your bank account balance is simply Stored as a column somewhere in a relational database No, right because what if you disagree with your bank? You say I should have a thousand dollars in my account and your bank says well the column says that you have a hundred dollars Sorry Well, of course like that. That's not how it works. In fact your account balance Is an equation? It's taking all of the transactions all of the debits and credits that have happened over the lifetime of that account And just summing them up and you can do this manually, right? You can print out your account statement You can go and add and subtract all the debits and credits and you can figure out what your bank balance is using that and event sourcing is actually The same thing just like your bank balance Which can be calculated and recalculated at any point all of your current state is transient in an event sourced world Your source of truth are your events your transactions not the balance or the current state so Before I go any further. I want to talk a little bit about what I mean when I say event because this is a very overloaded term in our industry In the context of this presentation when I say an event I mean there's something that have happened there are of significant interest to you to the business their business fact These events tend to be expressed in the past tense because they are something that have happened, so Potentially making a fool out of myself If I was working in a bank domain, I might have events that look something like this new account application received Custom identity verified account opened funds deposited We're drawn transferred account statement issues these is this is what I mean when I say events Right. I like to use calendars as a good example for event sourcing as well A calendar domain might have events that look something like this appointment schedule the appointment reschedule the invitations that have been extended since accepted or declined and And of course you can cancel appointment as well, right? Call tramp or I work. We are a Employee engagement platform. We help companies collect understand and act on employee feedback Events within our domain would look something like this employee added question description adjusted Survey launched question answered Additional comments added Survey closed report shared action item committed and so on The point is not necessarily Exactly what you would name your events It's that these events in an event source system are the absolute center of your domain and modeling them well is crucial And when we talk about modeling We usually talk about these kind of things, right? Let's let's say we're building an e-commerce website We're building a shopping cart. I'm sure that you occasionally go and buy stuff online In which case you're familiar with a shopping cart If I was to model that in a in an object oriented system or in a relational database or something I might have a cart object or a cart table that links to several line items And a shipping address and so on and so forth and and this could be anything right This could be a relational database. This could be a document in a document store It could be whatever and I think that we as developers have kind of been taught that the shape of this This thing here that the shape of it is the most important thing for us to worry about But this exact same thing can also be expressed in events. We can start off with an empty cart We can have an item that's been added to the cart and another item that's been added to the cart and then shipping address that's been provided and Using these events we can build this up, but what we end up storing are the events themselves not this Because in an event source world All of your current state is inherently disposable. It's a cash It's a cash that can be deleted or regenerated in in whenever you want to So effectively all it is is a different way of storing information restoring the events that have led to the current state Rather than the current state itself But it's something that has fundamental consequences That is an unusual sound with event sourcing you can still hear me in the back I hope even though there's obviously some drilling going on in the building With event sourcing all of your data is append only with event sourcing All of your data is immutable Append only in immutable is the essence of an event source system So how do we deal with mistakes because mistakes happen? Well, how does an accountant deal with a mistake? Let's say that they go to transfer They want to transfer a hundred dollars from this account to another but they end up fat fingering a zero and they end up transferring a thousand dollars instead How would they deal with that? They wouldn't just go into the ledger and just erase a zero, right? It doesn't it doesn't work like that They would they would go and get the other party to transfer nine hundred dollars back Or actually they probably wouldn't even do that they get the other party to transfer the full thousand dollars back And then they can do the transfer properly they are they're correcting their Their faulty transaction with additional transactions They are correcting themselves and in the event sourcing world. We call these correction events or compensation events Because if I add an item to my shopping cart and it turns out I don't want to buy it after all I remove that from my cart. I can capture that as an event as well I've Added an item. I've had another item and then I've removed an item before going through the checkout process And my question to you while I take a drink of water is are these two things the same thing? Who thinks they're the same thing? Who thinks the different things? The current state is the same Right, so my next question which is more of a hypothetical one. Can you think of two use cases? two different paths Like these two two different paths that lead to the exact same current state Yeah Excellent, so I have some bad news for you You're losing data if you have to use cases that lead to the exact same current state And you're not using event sourcing you're losing data somewhere along the way My follow-up question to that is how do you determine which data you're happy to lose? And can you tell me afterwards because I have no clue I? Don't know How to decide how to lose data I try never to lose any data because I don't know how to value this data I don't know what the potential future value of any data in my system is So I try not to lose it if I can help it and I think this is the main kind of benefit that you get from event sourcing Because I worked for CEOs that you know They wake up one morning and they have a thought in the shower and then they get into the office and they say you know what? People that have removed stuff from their shopping carts. They might still be interested in those things Right, they might still want to buy that thing. So why don't we do like a direct marketing campaign? To all the people that have removed stuff from their shopping carts and ended up not buying them yet, right? Now if all I had was a current state model, I would say absolutely we can do that We'll just add another table another thing where we start store to store all the line items that have been removed and Check back in six months. We might have some data for you, right? But in an event-sourced world, I would just say yeah cool We've got all of those those are events and we can use those events to build something up that we call a projection And we'll talk a little bit more about what that is The key point is that we're not losing data and while we're not losing data We can come up with new and interesting ways to interpret that data We also get a time machine time machines are pretty cool if we want to know This is a little bit out of date now But if I wanted to know what a particular report What how the system looked like a year ago? No problem. I Just replay events up until that point an event-sourced system is inherently deterministic You can recreate any state that that system has had at any point Often at this point in the presentation people go. Yeah, sure. That sounds great, but it can't scale. No way can that scale, right? Because what if I have a lot of events Well, that's kind of where the next part of the presentation comes in Who's heard of CQRS Who knows what it's short for the acronym I Command query responsibility segregation It's kind of an extension of another pattern called CQS, which is command query separation Because that's not confusing at all CQS was Initially an an an OO and an object oriented pattern which stated that a method in a class a method on an object Could either mutate states or return state. It could not do both You could either read data or write data. It could not do both It's separating reads from writes and Bertrand Mayer who coined CQS said asking a question should not change the answer CQRS is Very similar to CQS But I kind of think of it as CQS on steroids. You're taking the same concepts of separating your reads from your rights But at a much higher level Now I want to point out that it is very possible to use event sourcing without CQRS A lot of people conflate these two things and think that they're the same thing. It's not true You can also use CQRS without using event sourcing. They're two different patterns. It's just that they go really well together I'm gonna take you through and kind of build up what an event sourced system with CQRS might look like and In order to do so, there are four building blocks that I want to introduce to you We'll start with the event store so If your events are your source of truth, you should be storing them somewhere, right? You should have a database for your events. This database is called an event store An event store is usually almost considered a subsystem in its own rights I want to point out that there are both commercial and open source Event stores available out there But it's easy enough to just kind of roll your own and if you're just getting started That's usually what I recommend is roll your own And the simplest very possible Event store is actually super super straightforward. You can do it in a relational database like postgres mysql oracle etc It has a single event stream So if you've got a relational database think of it as a single table called events and It only supports three features the simplest event store only supports three features these three You can either put events You can get all of the events in order starting at an offset Or you can get all events in order Scoped by a particular something and we'll talk about what that something is later So how do we read? How do we use queries to read? Well clients and by clients. I mean a Mobile application web browser another system something that's trying to retrieve data From this application from this system. They want to read So that they can display information back to their users and clients. Let's be honest They're usually interested in current state, right? They want to you if you're trying to show that your bank balance to somebody you want to actually know what the bank balance is The event store is Not suitable for Presenting current state it just stores events, right? So we need something else that the clients can query to get current state. We call this a query handler It's a little bit of a detour right? We're not going straight to the event store. We're going through something else The way that that works is that the query handler effectively just sits and tails the event stream And uses the events to build up what we call a projection. So projection is some form of a of a current state model So that whenever a client wants to know something the query handler Uses the projection to return the data back a query handler will have many different projections each that are optimized for a particular query So whenever it gets a response, it will go and pick the data out from the right projection There are there could be hundreds of thousands of them and Each of these projections have their own kind of tailing process that I mentioned before that that that Tails the event stream and to build up that particular projection. This is called a projector The projector builds up the projection Is it worth noting if you're familiar with the observer pattern that this is not using that at all Instead we make use of this feature in the event store get all events in order starting from an offset So the projector will keep a pointer a bookmark to where in the event stream they're at What was the last event that I saw and processed and then we'll keep processing events and updating the bookmark as it goes Of course projections are disposable You can throw it away resets reset the projector back to zero and started from scratch again. No problem And they're built in such a way so that the projections and the projectors are completely De-coupled from each other you get some really nice benefits with that For example, you can use it to Just to use different data storage options for different projections if you like If you have some projections that are better suited as a relational database Sure, if you want to build up search indices go for it if you want to use a graph database a Projection could be as simple as just generating PDF files and storing them somewhere. That's a projection It doesn't like you can use whatever the point is that you optimize here for how you want to read the data Which is kind of different for From how we normally build applications today if you have an application that's built around a current state database They tend to be normalized But normalized data is optimized for how you want to write the data You want to make sure that there's no data duplicated and so on and so forth, but projections are often denormalized Which means that they much much much faster and easier to query They're cheap and they're easy to build Let's say we want to make a change to one of the projections. We don't actually change the projection. We just copy paste The projector spin it up as a new process build up a second projection We can look at the difference between the two and determine is this is this what we want And if so great, we just start reading from the new projection instead and throw away the old one What this means effectively is that you never have to do any database migrations in production ever again Which is kind of nice in summary Clients never query the events directly Instead they query denormalized projections that are optimized for how you actually want to read data These projections are built up by projectors that process the event stream These projectors are decoupled from each other and they don't share any states They're cheap and easy to rebuild Let's talk about commands Because in addition to reading clients usually also want to write It's not enough just to read information. That's boring. We also want to be able to change stuff, right? And just like we established that clients should never query the event store directly Nor should they be writing to the event store And the reason for that is that the event store is not responsible for validation So giving clients direct access to that is kind of dangerous. It's like opening up your sequel database to the world It's not a good idea Instead clients express their intention to do something in the form of a command And just like we had something that was responsible for handling queries We need something that's responsible for handling commands. We call that a command handler But kind of straightforward naming Sometimes naming is actually easy commands express intent Unlike events which are expressed in the past tense as things that have happened Commands are expressed in the imperative So open account Deposit funds withdraw funds transfer funds issue account statement. It's a thing that I want to do the difference Is that commands represent intent and they can be rejected? Events on the other hand there are facts and they cannot be rejected however a command that is accepted results in an event If I try to withdraw funds If I try to withdraw funds and that's successful that results in the funds withdrawn events So how does the command hand let validate events? This is where we start borrowing from DDD. How many of you have read this book? anyone couple of people like kind of I'm really I'm very sorry like those of you have had read it. I am so sorry It is one of the driest most boring books That you can possibly conceive of That doesn't change the fact that it is hugely valuable and that the information that's in there is a goldmine, right? There are some like distilled versions that I recommend But if you're up for it go to and actually read the book It changed the way that I think about software And here's the thing events kind of happen to something if I after this talk Go out on the street and get run over by a bus, right? I got hit I get hit by a bus That's an event that I would say holds some significance at least to me But that is an event that is happening to me, right? It's not happening to you. It's happening to me It Presumably also happens to the bus but the impact on the bus is probably lesser than the impact on me events happen to something and in the event sourcing world we call that something an aggregate and Aggregate is a is a term that we've borrowed stolen from the main driven design These aggregates become the nouns in our system So if we have these type of events in our bank again, I don't work for a bank you do I Would think that they would map back to aggregates that look something like this, right? your Account application received would map back to an account application aggregate your customer identity verified would map back to a customer aggregate Your account opened funds deposited and withdrawn events would map back to an account The funds transferred event would map back to a transfer aggregate and so on and so forth In our calendar domain, we only have three quite simple aggregates. We have the calendar itself We have the appointment. We have the invitation At culture amp we have the aggregates like employee demographic survey question response comment report action item These are our aggregates. These are the things that our events happen to Now these aggregates have identity that we need to track over time often They have life cycles an account application gets launched Created and then over time, you know, it either gets accepted or rejected and that could take seconds If you've got an automated systems, it could take a couple of days or however long, but it's got to finish It's got to start a beginning in an end Accounts might be around for a long time, but they can also be opened and closed right they have fixed life and These can of course refer to each other a transfer aggregate would You know link back to both of the accounts that are involved When a command is received, it is executed against an aggregate and it is the aggregate itself That determines Whether or not this command could be accepted or rejected and in order to do that the aggregate needs to know its current state If I'm trying to withdraw funds from my account the account needs to know if you've got enough money in it, right? So how does that work? Well, basically the pseudocode looks something like this You start by loading all of the events that have happened for that particular aggregate We load up a brand new aggregate instance. We replay all of the events To kind of reconstitute its current state and then we attempt to do the command which are the returns a true or a false It's either successful or or it isn't to do this we make use of the following Features of our events or we're putting events Obviously if the command is successful But we're also getting all events in order scoped by a certain something and by now we know what that something is, right? It's the aggregate ID the first thing that people often ask at this point It's like so every single time you're writing something you're loading a bunch of events and you're replaying that isn't that kind of slow The answer is it can be But it's only on writes Right one of the systems in particular that I Was building that was event sourced was 99.6 percent read heavy Only 0.4 percent of all the requests that it got were writes So we got all of the benefits on the on the on the read side And we actually had a read relatively low throughput of of commands that were coming through If you keep the number of events per aggregate low, it's usually not an issue In fact, I would say that an aggregate that Accumulates an unbounded number of events over time is is actually a bit of a smell This is why you really want to have these fixed-life aggregates, right? If I had an order placed events I would not put that on the customer aggregate because what that meant was that every single time a customer placed an order That customer would just keep accumulating events rather. I would have an order aggregate that links back To the customer Now the main modeling is hard and event sourcing doesn't change that Getting your events and your aggregates, right is one of the hardest parts of this in summary Clients never writes events directly Instead they express their intention to do something in the form of a command Commands are validated by aggregates, which is a concept that we've borrowed from DDD These aggregates fetch events from the event store and replace them to constitute their current state and If it accepts the command it results in an event So what does this leave us? We call this the circular architecture because it's a circle clients express their intent through commands that if accepted are Are Stored as events in an event store. There's a projector that tails the event store to get Events as they happen and uses that to build up projections and these projections can then be queried by clients Whenever they want to know something our reads and our writes our commands and our queries are separated and decoupled from each other They can scale independently writes if you've modeled your aggregates well are fast and always consistent. They're always used against current state Reads are very fast Because they're Dino they're using denormalized projections But they're eventually consistent and we'll talk a bit about what that means later First I want to talk about the fourth major building block react reactors So if commands are for client writes and queries are for client reads Their reactors are kind of for all of the stuff in between In order to illustrate that we need to extend the circular architecture a little bit See reactors they process events just like projectors do Just like projectors they keep a pointer a bookmark to where in the stream they're at But rather Than building up projections They react to events Either by triggering some form of external behavior or by emitting new events into the event store into the event stream or both So let me give you a couple of examples the first one I call racing events at a different level of abstraction, so We might have somewhere in our system a Bunch of orders being placed right What we would have is we'd have a reactor that subscribes to the order placed events and all it does is it accumulates a tally of how many orders a particular customer has placed When it detects kind of like according to its business rules the end Order from that customer might be the 10th or the 50th or the hundredth It goes ah cool. I know what to do. This is these are my business rules. This is my trigger and all it does When it detects that order is it pushes another event into the event stream and calling it Customer promoted to gold class. That's its job. It's just to watch the orders that are coming in and Knowing when a customer has qualified for gold class status This is a reactor that's stateful. It has to build up state It has to keep track of how many orders and any particular customer has placed and many reactors But not all of them, but many reactors are stateful The reactor would hold some form of reference data, right which we sometimes call internal projections All the reactor does is just watches the stream of events and reacts when its particular conditions are met Another example is undertaking asynchronous side effects. We may have a second reactor in our system that sits and and Subscribes to that customer promoted to gold class events that the other reactor created When it sees that It goes ah cool New gold class customer. I know what to do with that My job is to send them a welcome email saying hey welcome to gold class You're gonna get a little card in the mail. You might get some discounts or whatever And then it pushes another event into the event stream saying hey, I just sent the welcome email That's all it does sends an email and emits that event back into the stream What it does is it executes a side effect like sending an email? or generating an invoice or Sending some data to a Warehouse to fulfill the order or whatever it might be and then it says hey, I just did this thing Which means that we get an audit trail of everything that the system has done in the form of events I might take questions at the end if that's all right. Yeah In order to do this We make use of the following event store features. We're putting events We're getting all events in order starting at a particular offset Just like projectors reactors are decoupled from each other They typically run a single-threaded processes Because they're small and isolated and only do one thing like promoting a customer to gold class or sending one particular type of email Like projectors reactors are asynchronous and subject to eventual consistency They typically do not share any code They typically do not share any state with each other. That's what we why we get this kind of nice decoupled nature from They don't share any of their reference data their internal projections that they hold nor do they access any of the External projections that that the clients would query because again, that's how we get nice decoupling If a reactor needs state, it's responsible for managing that state itself And they should be built they can be built most of them are built So that they can be deleted or lost with no loss of system overall system integrity. So for instance if we if we lost if the If the reactor that sends out welcome emails goes down for a week We don't send emails for a week There might be some delays in that but the rest of the system will work Without a hitch reactors behave a lot like microservices if you're kind of familiar with that They tend towards They're not necessarily fully idempotent It's it's in some cases difficult to do but they tend towards it Which means that for the most part reactor will not retrigger external behavior when rebuilt against event history If we lose the bookmark For the reactor that sends emails, we just bring it back up and we'll go through the event stream It will not send emails again Which means that we can reset them to zero if we want to Reactors should do one thing well single responsibility and They encourage a business oriented decomposition of the system. It becomes really clear It's the gold class promoter reactor that has the business logic for when a customer gets promoted to gold class It's a business oriented decomposition of the system. I think they're really freaking cool But they are also where you find most of the complexity in your system As you have more and more flow on effects So in summary reactors process the event stream just like projectors do They react to events In several ways they can either raise new events at a different level of obstruction. They can trigger side effects They leave audit trails in the form of other events They're decoupled from each other. They don't share any state But they are often a source of complexity So quick recap Event sourcing makes you store business facts as the source of truth for your system These facts are immutable and appended only Event sourcing inherently makes your system deterministic CQRS, which is a different pattern in the circular architecture works really well with event sourcing Clients express their intentions to do something by a command which if accepted results in events asynchronous projectors process the event stream and build up denormalized projections Which clients can query whenever they want to know something and We have these asynchronous reactors that Also process the event stream and react to events according to the business logic outputting more events So what's the upside why don't you want to do this? Well, there's a few different reasons The first one that I think is like I feel like event sourcing inherently is treating The core of your business with respect. I Don't think that you can place any higher or more respect to your data than placing it at the very very center Of what you're doing and then leaving it alone right never changing it These events they represent fundamental business happenings that Always have been and always will remain of fundamental interest to the business We will always care about our customers and our orders and our accounts and whatever things we have in our domain They we will keep caring about those things We're not losing data that we can't value because we can't predict the future. I have no idea What crazy ideas? My CEO is gonna have in the shower in the morning, nor should I have to know that right? But I am putting myself in a position where I can respond to those things. I Also think that event sourcing inherently supports rapid iteration and reinterpretation So here's the thing I've got something that I have to admit to all of you. I have never Worked in a system that gets easier to change with time Has anyone here ever worked in a software system that gets easier to change over time? I have never met anyone Who has worked in a software system that gets easier to change over time never Sorry If you decommission it sure at that point, you know, yeah, whatever. Yeah So the thing that I've been thinking a lot about for the last few years, right? Okay, if I can't make it easy to change over time, how can I Change the cost curve, right? I don't want it to go like that because it will get very very hard very very quickly How can I keep the cost of change as low as possible for as long as possible? How can I do that? Right and my hypothesis is that event sourcing lets me do that and Maybe that seems antithetical to you right How can something so rigid as immutable append only data? Enable change. It doesn't make any sense well Here's the thing just because some things change quickly doesn't mean that everything does If you work in a bank, right, there are key concepts in your bank that don't change all that much Accounts don't change Like they do but very very slowly, right? Funds that are being transferred like these are concepts that don't really change and If we design these things well, the rate of change is actually quite low for our events But it's the reinterpretations How we look at that data how we experiment with that? That is easy to change saying we can get rid of when they're no longer relevant Because here's the money pitch for me event sourcing makes you forces you to separate recording of what happened from interpreting what that means Right and order was placed great That's a fact. That's something that happened But then we have fulfillment processes and we need to generate invoices and email them and we need to promote customers to goal class and These are interpretations From something that has happened and those are the things that change our order fulfillment process will change the way There are invoices look will change our business rules for promoting customers to goal class Those are the things that change The records they stay the interpretations are the things that change I think event sourcing is kind of inherently Compatible with reality, which I think is a good thing Event sourcing we kind of seek to emulate the real world as much as possible and in the real world Core concepts change very slowly as well. I think about cars, right? Cars have been around for what 100 years 150 years something like that How much have they really changed over that period of time really? Not very much right Yes, we frantically Innovate at the edges of a car. I we change the braking system We move from one type of fuel to different to electricity. We we but like fundamentally a car still has four wheels It's still got a steering wheel. It still has all of these things. Yes We innovate with bits and pieces, but as a core concept it moves really slowly. I Think that event sourcing where you you know you have your core concepts And then you frantically do things that the edges of your system is kind of just like that Just like in the real world time is a first-class citizen I really wish more systems had time as a first-class citizen and not just an afterthought In the real world the past is done And if you don't like it, I Mean sorry, all you can do is compensate for that, right? I've made some bad jokes in this presentation I can't take those back. I can't undo What I have done all I can do is compensate for it probably by making more silly jokes, but my point is we Can't take an eraser to the real world just like we can With many of our software systems, and I think that that is actually a good thing The real world is eventual consistent as well because there is no central control The real world does not have distributed transactions across multiple autonomous agents That keep them in sync doesn't happen, right? The real world doesn't have that No, the real world is made up by independent actors That do the best they can with the information that they have and that is what we're seeking to build in these systems as well I think event sourcing and CQRS encourages some good practices and supports some kind of common quality attributes You get nice clean separation of concerns through isolated asynchronous components If a part breaks down it only affects itself when it comes back up it picks off where it left off New features in my experience having worked on dozens of these systems now tend to evolve Extending code you adding new code. You're not changing the code. That's already there You might be deleting old projectors or old reactors, but you're replacing them with with new ones You're extending code. You're not changing that stuff. That's already there It encourages code that is expressed in business and user terms Like item added to shopping cart rather than cart and line items It removes these abstractions and It forms a ubiquitous language and I cannot emphasize enough how much business owners love this in my experience the people that Really really embrace event sourcing most are the product managers the business analysts Why because they already think in This way, right? They don't think of their website their e-commerce side as shopping carts and line items and Shipping addresses they think of it as a user logged in they added something to their cart They added something else to the cart that went through the checkout process We're removing these abstractions and we're forming this ubiquitous language We are meeting them where they are already at It's hugely valuable It also provides logical seems for services you slice by capability eventual consistency Is inherently more scalable and available than approaches which mandate real-time consistency? reads can scale horizontally and achieve true high availability Right can be very very fast. Well still in many cases actually remain single-threaded with all of the benefits that you get from having single-threaded writers in your system, right? If you're integrity minded if you want to make sure that you can never ever lose any data for any reason like use worm media So worm is short for write ones read many Think a CD-ROM, right? You write the data once and then you can read it many times But you can't actually overwrite it. You can't change it once it's there A pen only data of course Ah provides inherent Auditability These systems also avoid some common pain points that we might have we talked about data means migrations. They gone forget about them There is not a single ORM in sight There are no tightly tightly coupled lazy loaded object graphs there are hard to understand and reason about and From the way that I've worded this I'm sure you can tell that I've got some particularly bad experience with this Right It lets you use the best tool for the job These different building blocks their contracts is the event itself, right? So you can use different tools different languages if you so want to you can use the right data storage for your projections graph databases Search indices, etc. You're not constrained in the same way But they also provide seams for team organizations. So the way that We're Structuring our teams at culture amp is like we've got the event generating teams and the event consuming teams Right, we got the teams that are focused on how to read the data I'll write the data and then we have teams that are responsible for the interpretations And you can use event sourcing for parts of your system. You don't have to build up the whole system like this, right? You can have certain models within your system that are event sourced and other things that do whatever You get a free time machine, which we've already talked about. It's kind of nice So what about the downside? Because it sounds pretty good, right? At least I go and like, oh, yeah, this does pretty good. There are some significant downsides. However, though First one I call handing over the crown jewels, right? So here's the thing like there is nothing more important than the data that we store And the data persists even as technologies are replaced Who here works for a you may not have been in this company for this long ride But who here works in a company that's been around for more than 10 years? Really one person works for a company that's been around for more than 10 years Come on, race race your hands people. I know it's like Who here works for a company that's been around for more than 50 years Yeah, if you're working the bank, that's usually the case, right? What about a hundred years, right? Your organizations will have data that outlasts whatever technology that you're currently using You have data in your organization that was created Well before the technologies that you use today were even invented the data persists. The technologies change. They're replaced And so in any organization fundamentally changing the way that you store data is Actually a risk. It's a huge risk And this is what we're talking about. You're fundamentally changing the way that you store data And as the VP of engineering if one of my engineering teams came up to me and said hey Can we fundamentally store the change the way that we store data? I'm gonna say no and The default answer to whether you can fundamentally change the way that you start data should be no With events sourcing mistakes are forever, right? Pendable immutable Natural evolution of event schemas do happen and you have to keep supporting Old event schemas over time. You can't change events after the facts, right? You have to create compensating events correction events rather than just going in and fudging the data, right? That's both an advantage and a disadvantage, right? It's good that you can't do that, but it's also bad that you can't do that because there are times where there would be a hell of a lot easier Confidentiality can be an issue. I Don't know if GDPR is something. That's that's been a topic for anyone here It's been my life for the last six months like If you get a request to delete data like Mutable offend only right shit Look, there are strategies, but these strategies can be very painful and Some of them are not for the faint of heart. It's possible But it's a little bit more difficult The next issue the next downside is opacity, right? Relational databases are great because you can just log on to one and have a poke around, right? You can go and just have a look at your data if you want There are GUIs for it basic SQL skills, and it's not just developers, right? I've worked in organizations where a business analyst or a Product manager or whatever has got some basic SQL knowledge they connect to read it up the guy where they can't do any damage But they can just go and poke around an event store renders all of these tools all of these skills all of this knowledge and all of this access Completely irrelevant and this is a big deal because your data is way more opaque in an event store than it is in a relational database You have to create projections in order to make sense of the data You have to and then there's complexity I've got a bunch of side notes or sort of side rants About a lot of things. It's like I've got keywords that if you say this keyword I launch into a rant There's one of these side rants that have made its way into this presentation, and that's the side rant about complexity I often come across developers who says, oh, you know We should be able to build simple systems that don't have any complexity and I'm going no you shouldn't Your business is fundamentally complex. Therefore your computer systems have to be complex as well, right? Your systems are complex because your business is complex. You have that inherent necessary complexity If you don't have that in your business if you don't have a complex business Then you need to get the hell out of that business Because because somebody else is gonna come and copy it Right. You need that complexity But don't confuse that don't conflate that with the accidental complexity That we add to stuff as developers, right? You have to be able to separate what's necessary Complexity and what's accidental what stuff is just kind of there that doesn't need to be some does and some doesn't But we can't avoid it and if we can't avoid it then we can choose how to implement it I'm gonna Transition to the next slide in a second, but before I do I have to give a disclaimer I'm one of these people who believe that All models are wrong, but some are useful Every analogy that you use are going to fall down under some scrutiny after a while So just bear that in mind Hit hear me out. Hear me out. I Think that monoliths in some way is kind of intended to be a simple arrangement of complex things Right what I mean by a simple arrangement. I mean like it's a thing. It's a single thing. It's a unit that you deploy Right. It's a sink. It's got a single test suite. It's got a single thing It is a simple arrangement But it's when you take a peek under the hood where the complexity comes in You've got all these dependencies that are going in different directions sometimes in circles Don't get me started on circular dependencies. That's another one of my side rants It's the things under the hood. That's complex Event sourcing with CQRS on the other hand is meant to be the other way around, right? The things the projectors who react just does the aggregates are meant to be simple It only thing it does is promote customers to go class when they've got X amount of an amount of orders the only thing that this reactor does is sends an email the only thing this projector does is build up a Projector take projection tailored for a particular career. They're meant to be simple But it has a lot of moving parts. That's where the complex arrangement comes in and my caution is Beware of devolving into a complex arrangement of complex things Right You're with this type of system. You've already got the complex arrangement. That's the part that you can't avoid That's the that's the complexity that you get But don't build the things complex within it then, right? Don't make your reactors too complex don't make your projectors too complex This might seem like Not a hugely weird thing, but it's actually the number one issue that I've encountered in this world And it's really really easy to get here Because this is a fundamental paradigm shift for how you build Applications if you're taking this to a team that's used to building monoliths. They used to building complex things Right now then you getting them to build Those same complex things in a complex arrangement Now as mentioned like business take-holders, they love this stuff We think that you know the code is expressed the way that they think about it BI analytics people love it because they get nice event streams that they can pipe into their data warehouses and So on and so forth Surprisingly ops people love it too You might not think that but like it is actually really easy to back up and just Recover from disasters when you all you have is an immutable append only data stream, right? It's really easy to distribute it across multiple Like reads when all when you when you don't have them any multi multi rights Scenarios They really really like it, but it's developers those pesky developers. They're the ones who struggle with this thing, right? Part of the problem is that it's not mainstream. It's gaining traction. I think but it's not mainstream There are not enough resources Books courses presentations like I'm obviously trying to do my bit, right? But that might not be enough and there may never be enough I Used to have a slide in this presentation that said that there aren't enough libraries and resource or frameworks out there I've changed that now to be very very careful of all the libraries and frameworks out there a Lot of them solve the wrong problems along in the map abstract away the wrong things. I have not seen a team Picking up a framework or picking up a library that does event sourcing be successful The teams that I've seen if they haven't done it before I should say the teams that I've seen be Successful are the ones that have taken the time to learn the whole paradigm themselves first It can be difficult to hire Like Teams that don't have the prior experience are the ones that are going to struggle You almost have to have somebody who's done it before and we don't grow on trees, right? It's a catch-22, right? I've seen teams that do this once and they screw it up and then they never try it again Like it's really easy to get there Then this the Jacob of productivity Has anyone heard of the Jacob of productivity? It's my favorite thing in the world. It looks like this right you start here and Then it goes down and then it goes up things will get worse before they get better It takes some time Before the benefits kick in you are making an upfront investment It's super easy to just you know go Ruby on Rails build my application for me right scaffold all the things It's super easy to do that. There are lots of ways to build an application very very quickly. This takes longer But it's that keeping the cost of change lower for longer That's the idea right so you get the benefits, but you get them after six months or a year or five years or ten years That's when the benefits come in This is particularly true if the decision to use event sourcing was not made by the team itself I've found I've given this talk at a couple of Architecture conferences where I sit in front of a room of people who don't actually write code But who tell other teams how to write code, right? And my message to them has been do not impose event sourcing on a team that doesn't have the buy-in That doesn't have the stakeholder support to deal with the fact that things are going to get worse before they get better And most importantly that do not have the intestinal fortitude to deal with the issues that come up as a result and then there's that last thing that Sheetie thing that we're not supposed to talk about but we will anyway, right? So commands are synchronous and that's awesome. You issue a command and you immediately get not immediately But synchronously you get an act or a knack, right? It's either was successful or it was unsuccessful queries are synchronous as well. You ask you get a response The projectors are asynchronous, right? What that means is that after an event is has come in It can take some time for that to show up in your projections Which is kind of shit? Reactors, they're asynchronous too. Like all of these cascading effects. They're also asynchronous, which is kind of double shit And because all of your projectors and reactors are autonomous and decoupled from each other Those changes can show up in different times in different parts of your application, which is like all of the shit, right? We're not all parts of your system has the same idea of what current state is That is called eventual consistency Now here's a thing That a lot of people might not realize The real world is eventually consistent in the real world independent actors can act without consulting each other And if you look for eventual consistency, you'll find it everywhere so I was told and I gave this talk in Hong Kong on Tuesday and they and They told me this thing about Singapore So you'll have to correct me if this is just Hong Kong by Banging on Singapore or something But apparently there's a thing here where you you take your phone and you put it on a table to kind of reserve your spot Is this true? Not your phone, okay, that makes me feel a little bit better There's some people that do it, right Yeah, let's let's say for the sake of the the example that I'm about to use this thing is true, right? Yeah You leave your your phone at the table you walk off to get your food While you're off getting the food Your food might get or the phone might get stolen, right? That's that's what I worry about, right? Apparently this doesn't happen in Singapore, but However Now here's the thing your copy of the real world while you're off getting the food the phone is exactly where you left it, right? It's exactly where you left it Because yeah, your your your copy of the world is not in lockstep with everything else Minutes later when you get back to your table you find out that it's gone And that's when your copy of the world is made consistent With the rest of the world, right? This is eventual consistency and if you lose your phone, right? Yet some point you have to tell your family, right? I don't have my phone anymore That's when their copies of the world are made consistent If it's a company phone you have to tell your company, right? That's when their copy of the world is made consistent You might have to call up your phone provider and say hey Can you block this this sim card or whatever? That's when their copy of the world is made consistent your insurance company, etc etc All of these people who care about whether your phone is in your possession or not Right, they find out at different times. This is eventual consistency And here's the other thing your system that you're building is Already eventually consistent You might not think that it is but it is right the second to split second you let go of a database connection and and tries to to to Formulator responds to return back to the client whether that's html or json or XML or whatever it is that data could change Right, you may have used as the look on at their screen on their phone or in their web browser and the underlying data has changed Your system is already eventually consistent. You cannot avoid this You can narrow the time frame as much as possible. Sometimes it is literally microseconds But you can never fully Get rid of it and if you can't get rid of it if you can't eliminate it you have to manage it I find that modeling lessons from the real world is very very helpful when dealing with eventual consistency because risk is a function of time Who here thinks that in a nanosecond of eventually consistency? That's okay one nanosecond Yeah, a few people like yeah one nanosecond. That's fine. Yeah, what about a month? What about a month of the of eventual consistency? Is that okay people like no no way? Oh? My response today is great awesome. That means that somewhere in between one nanosecond and one month is your acceptable level Threshold of it of eventual consistency great. We just have to figure out what that is right? Of course this threshold is contextual If I'm showing the order status back to a user that just placed an order I probably want that to show up within seconds right because otherwise they might they might go oh shit I can't see my order It might not have come through. I'm best go place that order again, right? I don't want them to do that They're making a bad decision because the data that they're looking at is out of date Probably want that to be sorted within seconds But my CEO is not gonna fundamentally change how we do marketing If the sales report that he looks at are a few hours out of dates, right? So with I'm generating sales reports eventual consistency of a couple of hours is probably okay, right? All of this is a business risk. It's not a technical one and The question you have to ask yourself is what is the risk of making decisions bad decisions With out of date information. What is that risk now? There are ways of managing eventual consistency five main ones The first one is you educate your users you say hey, thanks for placing the order Please allow for a couple of minutes before you get the order confirmation and all that sort of stuff, right? If it hasn't shown up in an hour, that's when you go and place the order again if you've ever unsubscribed from an email newsletter and you get the Please allow for seven days for this to take effect that is somebody trying to educate you on eventual consistency right If that doesn't work for you if you don't want to do that you could also lie to your users What I mean by lying is if a client makes Sensor command and that command is synchronously act right the client knows that this thing was successful Right, you might not show up in the projections yet But the client can cash that data locally and display that data back to that user Right Nobody else will see that but that one user will This is called reading your own rights Right You're reading your own as you're writing you're keeping hold of that data because you know it was successful You know it will show up eventually You could also build new user interfaces that that emphasize the eventual consistent nature of the world Don't redirect people back to the page where the updates will show up two minutes later Find different workflows and paths build different user experiences You can slow down the user Works as well. Let's just show them a spinner Until the data is up to date in wherever it needs to be right you can do that And of course you can speed up the back end as much as possible to narrow that time frame. I Have seen all of these Used successfully To manage eventual consistency with insistence. Yes, even lying to your users. I have done that. I have no shame Though eventual consistency is Manageable for many businesses. It can be an absolute blocker event sourcing Is a pattern. It's not a silver bullet. It's not the answer to all your prayers. It's not The thing that's going to solve all of your problems but it is proven and Has been for hundreds of years and when I named this talk I Was shopping around talk titles and I named it, you know Go back to the future with then sourcing and secret and I shopped that title around and I had People saying that's a stupid title for for a talk. What are you gonna do? Like you're gonna have to add influx capacitors and Pop culture references and all that sort of stuff mighty McFly all it like I'm like, yeah But it fits so perfectly right because this is a pattern that we have used for such a long time and maybe maybe this Can actually be a key to how we build software systems in the future And for me it all comes back to this, right? How can we keep the cost of change lower for longer? This is my bet, right? This is the bet that I'm placing not for every system that I build This is doesn't work for every system. It doesn't work for every problem, but for many This is the bet that I'm placing Keep the cost of change lower for longer because I think that this is what we all struggle with as Software developers we talk about technical debt we talk about Refactoring and all of these things that make it more difficult to Or ideally makes it more easier to do new features, right? Keep the cost of change lower for longer. Yes, there are significant disadvantages with this these patterns But if you can overcome them, it can be incredibly powerful I like to finish with a quote from one of my previous team leads So so one of my ex-colleagues past team leads Who said event sourcing is kind of like a multiplier? If you do it well, if you do it well, it gives you almost magical powers in terms of flexibility and scalability But if you do it poorly, it will multiply that as well, right? And your system will become worse Yes, worse than even the crappiest balls of mud and with that. Thank you very much for listening Are there any questions? Yes So the question was I'm repeating it for the for the camera The question is why is a reactor separate isn't it just another type of command handler command handlers receiving put from users? So a command is always a user initiated Whereas a reactor it it's literally a trigger for business rules It's a user that listens to the environment. Yeah, I can kind of sort of buy that I Think of them as very separate things though one is So the benefit that you get from the separation is kind of like so I've dealt with a lot of systems that have these kind of collaborator objects or service objects or use cases Whatever they call right and it's kind of like you get an order and it's a collaborator object that determines All the things that has to happen as a result of this order, right? We have you have to go and generate invoices and email the invoices You have to go and notify print shipping labels and send that to the warehouse You have to go and do all of these things promote customers to go class and so on and so forth These collaborator objects over time become very very difficult to change. They become the complex thing within your system By doing this type of architecture all you're doing is placing an order and you just have to validate whether you can place that order or Not and if you do you capture that as an event and then all of these side effects all the consequences as of that order Are kind of like distributed out over the world? They're done asynchronously you have these independent actors their reactors that you know You have one that's responsible for printing the invoice or sending the invoice You have another one that's responsible for printing the shipping labels You have another one that's responsible for promoting customers to go class and each one of those become very very simple and If you know, you know if you want to change one of those things it becomes very easy to do is the theory Presentation Transactions like database transactions You often build events or systems without a single database transactions anywhere Which they're not incompatible like you can have reactors that are Committing multiple events like the reaction event and then it sends multiple events to back to the event stream That can be done in a transaction You could have for instance command handler that you know the when I say a command results in an event That's a simplification. That's a generalized session. You can have commands that result in multiple events You might want to wrap those in a transaction to ensure that either all of them go through and none of them go through, right? So there are certainly use cases where you might want to have transactions But you don't need them and I don't necessarily see them as incompatible as such But they are they are definitely in many cases not required Depending on your database your database will use transactions anyway even for a single single rights No, usually not If you want to make you withdraw Yep Sure Yeah, so so that That's a different problem for me. That's a race condition, right? You have two concurrent processes That are both trying to do the same thing at the same time now Or an airline booking or whatever So There are a couple of ways of dealing with that in this world the first the easiest one Which is what I mentioned earlier You can do that as well But the easiest thing is actually to keep your command handler single threaded Because if you do that you'd have no concurrency issues right There's only ever one thing that can happen at any given time and I've seen people do literally that or they shard out aggregates I That is that is something that you have to acknowledge and deal with and that already exists in your current system There are the ways to deal with it you can use optimistic locking on your event store There are there are lots of ways to deal with the concurrency issue if you need the concurrency It is possible, but yes, it's it's kind of a downside Yes Yep Yes You tried to relate for example incidental for example when Google migrating from Penguin to Panda And then finally They feel that doesn't there's no competition and get proof to go wrong They go into the eye. Yep. Now the question is that until now The cash cow is still belongs to them because there seems to be no contenders we challenge that capability you have that capability You think that you would be able to challenge that because you know why when you give an incidental for example a Singapore context to that of tissue paper, which is data form How do you correctly the kind of lies that Donald Trump mind at work I Don't know that I have an answer to that I I Every single time I have to admit every single time I go through this presentation I say events are facts and facts cannot be rejected I always think about Donald Trump because he consistently rejects facts I Don't I don't have an answer to that question. It's a it's a good one, but yeah, I'm not sure Donald Trump lies to his users. Yeah I would Yeah Which book sorry Yep I See By innovation by two manufacturers but based on 40 pages And you know and many more. The kind of, you have the acronym, you illustrate it, because you know why, that was many years ago, and I don't know, the guy is quite very smart, and I don't know whether you know him or not. All right. Yes. In the command comes in, it has to accept the rejector. At the time, it has to aggregate from the events tool. Yes. It has to be like, can it reuse the state of the project or it has to do it from the start? Yes. So the question is, when an aggregate receives a command, does it have to go back and replay every single time? The answer to that is no. I used to have slides in this presentation that talked about that. Obviously, the presentation is already long enough as it is, so I had to cut something. There's a pattern called the snapshot pattern, which is basically using the mental pattern, and what you're doing is you're having the aggregate cache its state as of a particular event. So it's not caching the current state, but if you consider that an aggregate has a thousand events, you're caching every hundred events, it's taking a snapshot. So when you need to load up the aggregate, you load the latest snapshot and then you only have to replay from there. Right? It can be faster. So if there's a... It can be forward, but it's... Yep. So your bank account, if you have got 10,000 transactions, you're not going to go all the way from like 10 years ago when the account opened and recalculate, but you might go back to last week, which was when your last snapshot was made, and just look at the transactions that have happened since then in order to calculate your current state. So you can persist... You can persist the snapshot, but that is also something that you can throw away, right? And rebuild. Yes. You mentioned how the event log is immutable. Yes. And once it's back there, you can't get rid of it. Yes. But you also brought up, for example, things like the GDPR. Yes. So the question was around, if you have a legal requirement like GDPR to actually delete data, how would you do it? There are strategies, I mentioned that. The short version is if you have to delete data, you delete data, right? How you do it or how I do it is I redact the events. So I don't delete the event, I redact it. Yes. So that's the first thing that I do. First thing I do is redact because I don't want to leave gaps in my event stream. Like I don't want any consumers later down the track to see that, oh shit, there's a gap in the stream here. I would want it to know that there was an event there, that's no longer there. So that's the first thing that I do. The second thing that I do is I capture a new event saying I just did that. So a redaction event. A redaction event. And that is because the fact that I redacted an event in and of itself is a fundamental business fact, right? That's of interest to the business. We have deleted some data. That's of interest. And that's what tells downstream consumers that hey, if you're holding on to a cache of this thing before, that's their trigger for deleting that. Does that make sense? Why don't you store pointers to the object and delete the object? Because the pointer exists, but the object will not exist. Let's say you store the customer detail in a separate table in a separate database. Delete that item in the database but the pointer still exists. You do that? You can. I feel that that type of system is inherently more complex than just having a single event stream, though. So you could do that. And I've seen that done as well. Not quite that way, but similar. Yes, it is possible to do that as well. Would it be complained to the GDPR if you delete the object? Yeah. That's pointed. As long as the PAI, the personal identifiable information, like I'm not a lawyer, but I would assume so. Because a pointer is like a UID or whatever. Yeah. Yes. Actually, I'd like to make a comment. The reason is because in forensic, whether or a dispute, you're not allowed to do all this. You're not allowed to what? You can't do that, though. You can't do that. I'm not allowed to do this because snapshot is your proof. Yep. And it depends on how many versions you go back and blah, blah, blah. Yeah. It's very complicated. You want to go into forensic, but I'm just telling you that these are exceptions and you don't need to do this. The right to be forgotten on the GDPR is not absolute. If you have a legitimate reason to hold on to business data, you can hold on to personal data, you can hold on to it in those kind of use cases. GDPR is a whole other topic in and of itself. The general point is that if you do have an obligation to delete data and you don't have a mandate to hold on to it for whatever legitimate reason that you have, there are ways of doing it. Through redaction and capturing that as a separate event is my solution to that. Yes. So once you build the event-strained event source, when you create credit and loss of money, it's easy for you to create a law in this topic. How do you keep track of what you can do when you go to the GDPR? Ah, the documentation question. This question very often. The short answer is you do it through, like there are lots of different ways of doing it. I've seen teams that use diagramming tools very effectively to kind of communicate the flow of data as it goes through the systems that understand each reactor and what the point was, that comes with the maintenance cost. In many cases, with good naming and good directory structures, you can go quite a long way with that as well. But yes, it can absolutely be a thing, but that's true for your monolith as well, right? It's true for any large, complicated system, is how do you bring on board new developers? How do you document any system? So there are absolutely ways of doing that. You pick your own strategy for how you deal with it, I think. The monoliths, you put fewer things to mess around. They are big modules. Big modules, yeah. Microsoft users and you get ideal models. Yes. When you talk about the conventional approach you're taking, yes, you see, you see climate normally has it intended as business as usual, but all the countries, businessmen, clients, they depend on things, on the defense factor, the circumstantial factor. The question is that when you, you see, you eradicate the fear factor, do you insist it on the basis of, you know, revolvable versus re-engineering kind of approach and position? I'm not sure I followed that at all. I'm sorry. Because if you want to touch and attack, transformation is one thing. Yeah, great. And transformation is yesterday's transition. Yeah. And tomorrow's transition is convention. Okay.