 Can you hear me? OK, I guess we can get started. Thank you very much for coming. My name is Vojtek Mach, and I want to speak to you about building an umbrella project. I work at Club Collect, and at my day job, I'm mostly working with Ruby on Rails. And on the side, I'm basically trolling the Elixir community with projects like the GitHub API adapter for Ecto, or the OOP library for Elixir. And today, I'd like to show another toy project of mine. It's Acme Bank. OK, so what's an umbrella project? Before answering this question, I'd like to take a few minutes to talk about some inspirations for this talk, some resources that I have found really insightful and interesting over the years. The first one is a talk by Robert C. Martin at Midwest RubyConf 2011. The talk is titled Architecture the Lost Years. In that talk, Robert described a typical race application to be just that, a Rails app. We look at the project structure. We see the app folder, and the controllers, and models, and views. But by looking at just the project structure, it's often a little bit hard to see what the project actually does, what are the important parts, what's important and what's not, and how the things relate to each other. One memorable moment for me from Robert's talk was when he said something along the lines that a top-level architecture should scream its intent. You should be able to look at it and understand what's going on. Another resource is a talk by Stefan Hageman titled Wrangling Large Rails Cold Basis at Rocky Mountain Ruby 2012. Stefan talked about large apps, and he gave a fantastic piece of advice about writing them. Never write large apps. Stefan showed how to structure a potentially big race project to be a collection of components, gems and engines. A gem is like a hex package and an engine. It's like a mini-race application that's often distributed as a gem. What I think was really innovative in Stefan's approach was that he used these engines and gems, not as a means of distribution of the code, but as a mean to break up your application into smaller parts, each with its own namespace and its own test suite and configuration and so on. And then the main race application didn't actually contain any logic of its own. It just held these engines and gems. Finally, I'd like to talk, I'd like to mention one other talk which is The Art of Destroying Software by Greg Young. The premise of that talk is pretty simple. Organize your project in a way that makes it easy to delete code. Perhaps there is a part of your system that is a constant source of bugs, or it's really hard to understand, or usually both. When you need to make a change in that part, instead of patching it up, what if you could just throw it away and start from scratch, start fresh? How liberating that would be. But you need to organize your project to make it possible first. And I think it's a really powerful idea to organize a project in that way to make it easy to delete code. OK, so what's an umbrella project? An umbrella project is basically a mixed file. OK, one second. Can you hear me? OK, thanks. No pressure. So an umbrella project is essentially a mixed file, an apps directory, and in that apps directory, we have the collection of the OTP applications. For the purpose of this talk, let's briefly talk about what an OTP app is. Per the documentation, an application is a component implementing some specific functionality that can be start and stopped as a unit, and which can be reused in other systems. An OTP application can have application dependencies. So these are the applications that must be started before this application starts. There's a way to configure it, and we can define the start and stop behavior. When we start some applications, what we want to often do is to start a supervision tree. And depending if we have these supervisors, we sometimes can call an application to be a regular application. Or if it doesn't have supervisors, it's sometimes called a library application. A good example of a library application is Poisson, the DJSON library for LXU. Finally, OTP application is kind of a standard. If you know how to configure an OTP application and start and stop it, if you learn how to configure one, you can pretty much configure any of them because they are all similar. It's a shared understanding of how we should work with them. The word application itself, I think, is pretty interesting here because outside of the Erlang and OTP sort of ecosystem, the word application has usually a bit of a different meaning. And when I think about OTP applications and the fact that it can be, again, configured, start and stop, a word that comes into my mind is a server or a service. And so when we have a big service, when we have a big application, a lot of code, we could call it a big service. But if we have a very small and focused one, we could have a microservice. We'll talk about microservices at the end of this talk. OK, so let's build. Let me walk you through building an umbrella project. So let's build the Acme Bank. So we create a new project. We're going to call it Bank Platform. And we pass the dash dash umbrella flag. It's going to create a little bit different mix project. In the mix file, we can see that we don't actually have the name of the application because the umbrella project itself is not actually an OTP application. It's just a container for your other applications that are in the apps folder. So let's create our first OTP application. And we actually have a pretty interesting decision to make where to actually start. We want to have a web user interface for our bank, but it doesn't mean that we absolutely have to start with the web functionality first. I believe in working on the hardest problems first. And so I'd like to start with building the core business logic of our application first and then keep it decoupled from the web stuff. And then later on, we would connect it. So let's build the actual bank application that's going to hold our logic. We CD into the apps folder. And we create a new mix project. We pass the SAP flag because we want to have supervisors because this bank application is actually going to maintain the storage as well. And yeah, if we create a new project within the umbrella, Mix will know that it's in fact inside the umbrella and it's going to change the mix file a little bit. As you can see here, a difference from our regular mix file is that we have the build path, config path, depth path, and the log file. And in fact, all of the applications inside an umbrella will point to these directories and files that are in the root of the umbrella. They share the same dependencies and log file and build folder and all of that. OK, so let's build the first feature of our bank. Let's compute the account balance of an account. A naive approach to that would be to have an account struck with a balance integer field. When we deposit some money into the bank, we would just increase the balance. However, this is not actually at all how banks work. And since this is really important to build a bank, let me do a quick accounting one-on-one. Any movement of money happens between at least two accounts. One account is debited and the other account is credited. For example, Alice transfers $100 to Bob. We would actually write two different entries for that movement of money. A debit to Alice for $100 and a credit to Bob for $100. If Bob transfers $20 back to Alice, we will have another debit for Bob and credit to Alice. And then maybe he sends 10 more. So we will have the appropriate entries. Now, to calculate the balance, all we have to do is to get all of the entries for Bob and sum them. A credit is going to increase the account balance and the debit is going to decrease it. So for Bob, sorry. So for Bob, we have $100 minus $20 minus $10. So it's $70. And for Alice, we have the opposite. I think I have my math wrong. Sorry. Oh, no, it's okay. So yeah, for Alice, we have the opposite. And this is actually a pretty important rule in accounting that the sum of all credits must equal to the sum of all debits at all times. And what we've briefly described is called double entry counting system. And double entry because we write and every transaction twice, one for each account. So you might be wondering like why I'm mentioning this. As I mentioned, it's kind of like the core domain knowledge of the bank, so it's kind of important. But also it's functional. The list of entries is immutable. We never go back and change an entry or delete it. We just keep going, keep adding these entries. And actually it has some interesting mathematical properties as well. The order of the entries doesn't matter. So it's both associative and commutative. Like it doesn't matter if it's a debit and credit or a credit and debit. And with some extra work, we could make it idemponent. When we see an entry, we just make sure that one entry is only calculated once for the balance. And with associativity, commutativity, and idempotence, this starts to look like a CRDT. And as you know, CRDTs are the foundation for the Phoenix presence. And there is actually a pretty interesting article on high-scolability website that basically says that banking systems are not strictly consistent, but are actually eventually consistent. And this idea of eventual consistency for banks actually is traced back to Renaissance era. And in a way, Florentine merchants like 600 years ago were using CRDTs before it was cool. So I think that's pretty interesting. And finally, the whole reason to talk about accounting, I think this stuff is actually non-trivial. And I think in most, if not all, non-trivial applications, there's always going to be this essential complexity that's specific to your domain. And I think it's very beneficial to separate it out so you can actually think about it in isolation, independent of some details like web. So briefly, I'm going to talk about entities in our bank. So we are going to have the account and entry that we just talked about, and then the customer. I want to keep a customer separate from the account because usually a customer can have more than one account, although actually there's just going to be one for one in this project. So in a Phoenix project, we would implement these entities as models, so web models, account, and so on. However, since yesterday, as you know, models are out of fashion. And moreover, it's not actually a Phoenix project. So we are going to put this in Lib. So it's going to be Lib, bank, account, and so on. So account and entry are a little bit different than a customer because they have a different rate of change. When we implement our double entry accounting system, like after we implement it, we should be pretty much done with that code because the accounting rules are very stable, like 600 years old. And it's like, and so that part is very much different from the customer because a customer is something very specific to our application. The customer can change as our requirements change in our particular bank. And so I think it's very worthwhile to keep that separate. And in fact, I'm going to add it to a ledger sort of namespace and directory to sort of show that it's separate. So let's talk about persistence, how we actually want to persist this data. For simplicity, I'm just going to use Ecto. And now the question is, where do we put the repo access? So also, since yesterday, you know that putting the repo access in controllers is out of fashion as well. And again, we don't really have a, it's not a Phoenix project, so where do we put it? Since the bank ledger account and the ledger entry are in the ledger namespace, it's very natural to just put the extra functionality in the ledger module itself. So we are going to have a function to calculate the balance and a function to write down these entries and persist them. And we are going to do the same thing for the bank. So there's going to be create customer function to actually create it and then update customer and so on and so forth. So we have these three functions. And they are use cases of our application. These are the high level operations that describe what the application does. And inside these, we have the repo access or maybe calling out to different services. But on the outside, this is what our application is doing. And our additional benefit of that is that we can just call these functions in our controllers or tests or IEX or seeds. We don't need to go through the repository. We can just call these very short-named functions with very descriptive names and get the job done. So we talked a little bit about accounts and balances. And so let's talk briefly about how do we actually want to store that. So in many banks, we have to support multiple currencies. So we're going to keep both the amount and currency on the money record. We also will have some operations, like adding two different money objects. And we usually want to make sure that we add stuff in the same currency because you can have unexpected results otherwise. And also persistence. Since we store both amount and currency, we are going to need to serialize that to our database column to keep it in SQL. And so actually all of these things that we need to do are actually pretty generic. And they could be reused somewhere else. So I don't want to put that in the bank application. And I want to actually keep it separate. So then later on, maybe, if I'm going to open source it, it's going to be absolutely trivial to just publish it to hex. And by starting from the get go with a separate application for money, which is going to be a regular application with no supervisors, when I start with an OTP application from the get go, I'm more productive by doing that. Because if the code is already at hand in the umbrella project, I don't have to worry about versioning or releases or coordinating pull requests between two different projects. So it's much more productive. And it's actually a case for all of the applications in your umbrella. It's just so much easier to have everything in one repository, although nicely partitioned and have everything at your fingertips. In order to use the money application in the bank application, we need to set it as a dependency. There is an extra in umbrella flag that you can use. And then so MIX is going to know that it needs to find this package in the umbrella and not on a hex PM. We also need to add it to the applications list, even though it doesn't start supervisors, it needs to be there. OK, so let's talk about the web interface for our bank. This is how we are going to create it. We just, again, CD to the apps directory. And we create a new Phoenix project. MIX will know that we are inside the umbrella, so it's going to create the correct MIX file. And then we will, again, have to add a bank this time as a dependency to our bank web app. And this is kind of important. Our web interface depends on the business logic, but not the other way around. The business logic doesn't care about web. It doesn't reference it in any way. OK, so bank web. This is actually pretty boring, because since we already have the business logic, all we need to do is to just use it from the controllers and views. So just calling these functions. So I'm not going to show that. But I want to really mention, again, the separation of concerns that we've achieved by doing it. We can evolve the web application separate from the domain logic and the other way around. It's also kind of important when you have to make major changes. So I'm not sure if anyone was ever on a big web UI kind of redesign project. Anyone? So I've been on a project like that a couple of times. And I really wished that I would just start from scratch, because the UI was so coupled to everything else. But if we now keep it separate, again, as Greg Young said, we can just delete the web stuff and just start from scratch. Or at least we have a very nice sort of, we have good places to find the functionality. OK, so this is the application for the web UI and for the customers to use. And then in our bank, in production, we want to have some functionality for the operators, like customer service and stuff like that. So let's add kind of like an admin interface for our bank. I'm going to call it back office just because that's how it's usually called in the bank industry. All we need to do is to just, all we need to have here is just some screens to see all of the accounts and transactions and stuff. And I could write it, but there is actually an excellent XAdmin package on Hex. And it actually works out of the box with umbrellas. And in fact, I shouldn't even mention it because there is nothing special about XAdmin because it's just OTP applications. They just work by design because we are using, again, that shared understanding. OK, so with the back office, which is another Phoenix app in our project, we need to set up different ports in development. So maybe there is port 4,000 for our main Phoenix app and port 4,001 for the back office. So this sort of mention of different ports is a sort of sign that our deployment is going to be a little bit more complicated. And I'm going to talk about deployment in a second. But I also want to mention the talk Phoenix is Not Your Application by Lance earlier this year. I think having the back office and bank web in one umbrella project is a testament to this idea that there isn't anything special about Phoenix. It's not, in fact, your application because you can have many instances of Phoenix in your project. There isn't anything special about Phoenix. It's just an OTP app. OK, so briefly talking about deployment, I'm deploying it to Heroku just because it's easy. And in Heroku, you can only have one process combined to a web port, but we have two applications. So what I did is I have a very simple proxy application that just forwards the requests to the appropriate Phoenix endpoint, depending on the URL. And a big thanks to Gary Rini who allowed me to use that code. It's really simple. It's just a few lines, really. And speaking about deployment, distillery, which is an XRM replacement, it already has a first class support for umbrella. So if you want to do OTP releases with umbrellas, that should be very easy with distillery. So now, I want to briefly mention an authentication subsystem of our bank. So authentication is one of these things that pretty much always starts very small and often stays small. But there are certain cases where in certain apps, authentication can get really complicated. So I decided to start with a separated authentication subsystem first. So you might be wondering, well, maybe it's a bit of an upfront design. And I get that, but I think there is actually merit to it. If you are building a bank, we are going to know that we will have certain features that we are going to absolutely need. Like maybe there are different strategies, like the customers are going to sign in with the username and password, but the operators might need something like LDAAP. We probably are going to need two-factor authentication. We probably are going to need some tracking, like IP tracking, to get that annoying message that you'll bug in to your bank from a different country, and it's not going to allow you and stuff. So we have all of these extra functionality that we need, that we are going to need. So that's why I wanted to create a separate authentication. And in fact, the authentication subsystem is going to maintain the user accounts. And for that, there is a separate ECTO repo and actually a separate database, because I think that, in my opinion, each application should maintain its own state. So that's why there is a separate repository, an ECTO repository from the other bank. And thanks to Chris McCord for some inspiration for extracting out the authentication into a separate app. I also have a messaging subsystem in the bank, but I won't get into that today. And yeah, this is our bank platform. We have seven applications of bank, bank web, back office, master proxy, messenger and money. And each application is an OTP application with very well-defined dependencies. And because of that, we can actually graph how our platform looks like, what depends on what. So it's kind of like this top-level architecture that we want to have when we come into a new project. We can see what are the pieces. And as we evolve our project, we are going to have even more applications. So in a bank, maybe we are going to have some fraud detection subsystem that monitors the incoming transactions. Or maybe there's going to be a landing page that's going to be like a marketing landing page. Maybe it's going to become a CMS system of some sorts later on. So yeah, we can manage the growth of our system by keeping the applications separate. Okay, so I want to try a quick demo of how it is to actually work with an umbrella project. So let me... Let me go bigger. Is that readable enough in the back? Bigger? Okay. So yeah, so we are in the root of our project. We can inspect the apps folder. We can see all of our apps. We can run the tests. So if we are in a MIX project, MIX will automatically run tests for each application in your umbrella, which is really useful. When we CD into a protocol app, we can run the tests separately. So it's really good, right? We can work on our application separately, run the tests, iterate. And then after we are done, we can go back and run all of the tests to make sure we didn't break anything. Okay, so... First we have all of the code in one, directory in one git project. We can actually navigate the project very easily. So we can... Let's see. So we can go to ledger. We can go to money, which is a different OTP project. We can quickly jump around the project without going to a different repository, which is very fast to actually work on the project. Okay, so I have a console that I started for this project. And because we have these modules that describe our use cases, we can actually sort of look at them. So we have the bank, and then we can see there is a createCustomer function or createDeposit, findCustomer. So we can actually look at that. And again, like calculating balance, instead of calculating that in the controller, we actually have a place to put all of that logic, and we can document it. So, yeah. Oops, I should start it. So let me quickly show you how the application looks like. So this is the... This is a very simple user interface for the bank. That's running on port 4000, and we have the backoffice application of 4001. And, yeah. All of the code is available on GitHub. It's easy to double out to Heroku, and you can see all of the applications. So let's go back to the talk. Okay, so as we mentioned, an OTP application is a bit like a microservice. So there are well-known benefits and drawbacks of microservices. Like there are strong module boundaries and independent deployments, and the drawbacks are operation complexity. So I was planning to do a comparison of microservices and umbrellas, but if I told you that umbrellas give you pretty much all of the benefits with none of the drawbacks, you won't believe me anyway. So I will just leave it like that as an exercise to the viewer. So with umbrellas and microservices, it's a little bit like with microservices, right? They won't solve your problems. You still need to architect your system. You want to have just the right amount of applications and with the right responsibilities. You don't want to have too much or too many. And to conclude, keep an eye on the big picture. And don't write a large up. Keep it separated. Isolate it by functionality and by the rates of change of these components and delete code liberally. And that's all I got. Thank you very much. Do you have a question? In the front. So I'm kind of a big fan of microservices, but I certainly understand those drawbacks. Nonetheless, the big benefits to me are the independence, independence of technology and independently being able to scale or release different services. A lot of that goes away if you stay entirely in the Erlang Elixir ecosystem, but most of the time you can't, especially with large organizations, you're going to have all the technologies. So how do you map into that kind of a microservice environment? Yes, so that's a great question. I mean, if in your organization you just have components in other technologies, you just can't do anything about that. That's just the fact. But I think there is actually, I mean it depends on the organization, but you have to have it to stay on the same stack, because then you can switch to different teams. As far as scalability issues that you mentioned, I think it's still compatible with umbrella applications. You have the flexibility of deploying just one application to a cluster of nodes and other application to a different cluster. So you have everything in one project, but you can just figure out how you want to deploy it. You have a lot of options. So it's sort of like Google has this huge repository right with all of the code, but they still deploy it separately, so you actually have an option for that. Not sure if that answers the question, but... You talked about organizing your code in a way that makes it easier to delete. Are you just saying that an umbrella makes it easier to delete, or is there more you can say about organizing your code so it's easier to delete it? This principle of organizing your project to delete code is not specific to umbrella. There isn't anything special about umbrella. It's more of a general principle, but I do think that having an umbrella guides you into that direction, that you have these small pieces that are very isolated and independent of each other, and then you can take them and think about them separately, and then if you just want to start from scratch you can do that. But yeah, if you structure your big application in a way that each piece is separate, then you can delete that. But I just think that here with an umbrella it's a little bit easier to go into that direction where pieces are separate right there. I'm curious if you can speak a little bit toward whether there are any guidelines for how you would write integration tests for a system like this. I would also feel a poll toward the rabbit hole of having integration tests for every interaction between dependencies as well. So any guidelines there? Yeah, so that's a very good question. You can approach it like you would normally approach, but maybe you would have integration tests for the bank web, which is kind of like the entry point. You could have integration tests there, but I think another interesting approach is to actually even have... I mean, I explored with the idea a little bit, but an interesting approach is to actually have a separate kind of project that integration tests the entire thing. And what's kind of cool about that is that you can even deploy everything maybe to Heroku, like all of the applications, and then integration tests that kind of live, which sort of compared to microservices, it's a little bit easier because with microservices you kind of have to deploy everything separately, and here you have everything in a bundle. But yeah, I mean, I don't really have any specific guidelines at this moment, but yeah, I think there are good options. When you're using master proxy, is there a way to share logins and authentication? Could you repeat, sorry? When you're using master proxy do you have a separate administrative application? Is there a way to share authentication and logins between those two applications? Yeah, definitely. So yeah, there is the authentication subsystem keeps track of the login information, so you would use the auth application in either of these apps. And then the master proxy really does just kind of like a router. It just follows the request, so you solve the authentication problem on each separate web application. Anyone else? Thank you. All right, thank you very much.