 But we're talking about Phoenix 1.3 today. Show of hands, who has Phoenix in production here? Curious? Wow. Awesome. So nothing's going to break. So for those of you in production, but today we're talking about some of the kind of design decisions behind the new generators and some little bit of a course correction. And if you saw the Orlando talk, the first few slides will be review, but the rest of the talk is really all about the thinking behind that and how to think about structuring applications and why we're making these changes. So I work at Dockyard. If you haven't heard, we do Elixir and Phoenix. So if your company needs help building out a product, we'd be happy to help you, so check us out. And I have a little personal level up in my own life. I've been memed, so thank you, Luke and Jason. This photo was taken yesterday and I'm now, I've been memed, so it was my own kind of personal achievement for turning 30 this year. I became an internet meme, so pretty cool. But on another note, so a couple things are going away with Phoenix 1.3. You may have seen this slide. We are losing the root level web directory and we are kind of killing the term model anywhere in the code base and we'll talk about why this is happening. But I think this term has kind of just been more confusion, more frustration than it's worth. But with Phoenix 1.2 or prior to 1.2, we had a directory structure like this that Jose and I thought was a great idea because we wanted to send a message that, okay, this is an Elixir application, right? You have a lib and just when you run Mix Phoenix New, you get an Elixir app, you don't get a Phoenix app and we would create this root level web folder to send this message that the web part of your application isn't the domain of your application, right? So we chose this decision to send a good message. People saw this and they were just like, you're a framework and you're prescribing how I should run my application, you're generating something special, this isn't an Elixir app. So this ended up being like a backlash for people thinking we were trying to be extra special and taking over your application. So what we're doing now is something really creative is, well, we wanted to send this message, right? You have your app and you have a web interface but people got mad. So we hit the drawing board, did some computer-sized research and now we're gonna take web and we're moving it inside lib and now everyone's happy, right? Yeah, this was like the last six months of research. Yeah, so that's it, end of the talk. So it's funny because there's actually, there are other reasons behind the Elixir compiler why we had a root level web directory but it originally was to send this good message but there did need to be changes to the Elixir compiler to move web inside of lib because of code reloading. So those changes were made. So it was a little more complicated than that but we still wanna send this message and I think we can send the same message by having web inside of lib because lib is where you write your application code and if the web is part of your application then you write your web code in lib web just like you would structure anything else. So we can still continue to send this message and generate a normal Elixir application and the only special thing is we're gonna give you a root level assets directory. So now we don't pollute your root level project with like a package JSON or node modules folder. All of that regardless of what you wanna use if you're using Brunch, using Ember, Webpack, whatever you wanna use would just live inside assets and your 30 million node files and node modules will live inside assets. So as long as your asset builder can build your assets from the assets directory into priv, static, then it's fine. So this I think is a big one. And we're also introducing a web namespace. So a little bit more typing but again we wanna send this message of if you naturally namespace your modules based on your folder hierarchy anyway but when you see this, this is kinda reinforcing like okay this module application is the web layer. This is the web interface to my greater domain. So a little bit more typing but with A listening this is pretty much a non-issue. And kind of the bigger changes is around the code that we generate with Mix Phoenix Gen. So we've always said generators are learning tools and they're supposed to be like not the end point of your design but the starting point to try to kinda give you, get your feet wet, get up and running quickly and teach you best practices but I think we haven't done enough teaching in the right direction. So let's walk through maybe why that is. So with Phoenix 1.2 we generated code like this with Ecto so you could say repo all, give me all the comments. Like there's just two lines of code. It reads nicely, the database access like you know I'm hitting repo but that repo could be a gen server, right? That repo could be a net stable, could be Postgres database, it doesn't really matter. So there is some nicety here but it gets worse when we kind of look at more complex code. So when we also generated a persistence function in the controller, this is where we would kind of be hand wavy. We would say like well by default we're just gonna say you're gonna have a repo, you can insert some changes into the database and if you need like something more complex we just kind of hand wave and say well you can then extract that code and put it in a module. But the problem is what newcomers see is like okay I run this command and Phoenix put my code here so this is where I write my code and so it's not their fault, it's our fault because we're kind of sending that message and then it gets worse when there's no clear point where you feel like okay should I extract this? Is this too much? Is this too complex? So we can do a better job because if we walk through what would happen is people would say okay I wanna add this simple feature. I wanna be able to comment on Posts. So they'd say well I only have to add two lines of code in this controller so if I need to associate a Post in a comment I'll just add Ectopodesk, fetch the Post and I'm good to go. But the problem is we're now sprinkling these details of like how do I associate these data structures, how are they persisted, how do I fetch posts all in the controller. So if I wanna reuse this anywhere else in my application I have to know that I need to extract it but most likely people are just gonna duplicate this code. So we can already see like as soon as people start trying to extend the code that we generate we've sent the message like this is where you write it so then they just add a couple lines and this is like the ball of mud scenario where you think like oh I'm only adding two lines of code it can't be bad and then you do that for five years and now your controller create is 100 lines of code so we can do a better job here. So with Phoenix 1.3 we're gonna generate code like this where you used to say Mixed Phoenix Gen HTML or JSON comment where you would give a module we're not gonna require you to give a namespace module what we're calling a context. So you're gonna have to say Mixed Phoenix Gen JSON blog comment so you're gonna have to think a little bit ahead of your design so we're making you think okay like what is this comment system where does it live what's the purpose of this and then we'll generate code in the controller like this that just says some generic blog list comments like I don't care how they're being listed or fetched maybe it's from Prosgres maybe it's from an Ekto repo maybe it's directly out of an agent running some process and then likewise with persistence we're gonna generate some generic create comment function and this way we're gonna generate code by default that doesn't live in the controller the controller literally is just an interface web interface into your greater domain and then likewise when a user comes here for create they have kind of a nice rubric that they're saying okay if I wanna add this feature Phoenix has kind of pushed me into say okay I have this function already that can create a comment now if I wanna associate the post in a comment there is a place in my application where that lives so they can say okay I can fetch the post because I have a public function and interface to do that and then I can just extend the code that Phoenix has generated for me and you may notice here that I'm using the Lixars with expression so if you haven't used it much it's amazing but if you notice here we're not matching on like what happens if a blog create comment has some other return value then something that was okay like what if we get an error back with some invalid data but if there is a unique constraint violation and we don't have to handle that here so one thing especially with JSON controllers like a Docker we build mostly JSON APIs we had a lot of repetition in our code where it's basically like if you get an error back or error change set like okay convert the change set errors to a JSON message return the right HTTP response and you're done and then you do that in every controller and every other action over and over and over so we're introducing this new feature called action fallback it's a macro on the controller that says if your so your controller is a plug so if your controller action returns something other than a valid plug connection we're gonna invoke this other plug whose job is to take the connection before the action and the value that was returned and then convert that into a valid response so this gets rid of a ton of duplication and gives you some nice features so imagine here where I say okay create this comment and then if it was successful I can do the successful path but if there was some invalidation or any other error condition I can define this fallback controller that's just a plug so I can just define some functions that pattern match on the data types of my domain so if I can say in my domain if I'm using Ecto I'm gonna have a set of changes that could be returned and if I get error change set I know that I'm gonna put the unprocessable entity status and I'm gonna render the change set view and I can just do this in one place and then I can continue and just define more clauses for other data types of my domain so if I wanna standardize if my API boundaries define error not found as a return type I just wanna handle that in one place instead of 100 places in my controller I can just use a width expression use action fallback and say okay anytime I call into something in my domain that returns error not found I know that that's gonna be a 404 render the correct view and send the right response so for us this is especially a doc guard that's definitely removed a ton of boilerplate for HTML controllers it's not quite as useful you may be able to use it in some cases but there's often times where you have to put a flash message render specific templates but if you're building JSON APIs I think people will really like this feature so let's talk a little bit about why web models is going away other than this root level folder so I think part of the problem is this is the term model I think is wrong so when we took this decision it was like okay web models we've used in other backgrounds a lot of us come from Ruby but if we look here I was, client code was coming in where people would say hey we need this new feature you need to help us out or I would be talking to other people in the community that had larger Phoenix projects and as we were like pairing and going through their code we'd open the web models directory and it would just be like 50 files and the problem here is like you have no view in the application like if I look at this directory structure it's just a dumping ground like I can't really tell what this application does right it's like I can say okay you have users you have comments somewhere like we don't know how these are associated we don't know what depends on what so we're not sending the right message here like I want to be able to look at your app just open the directory and see like oh I can kind of see what this application does and understand the feature set so we wanted to get rid of this dumping ground and also the word model it just doesn't make sense because the entire point of your application is to model your domain right there's no reason to call some individual file a model it's like the entire point of the application is to model your domain and you do that with Elixir files that have modules and functions right so the Phoenix generators are gonna generate code like this where we can say when you say mix Phoenix HTML or JSON blog comment blog post we're gonna generate a blog ex file and then within that it's gonna have its own data structures right it's gonna say I'm gonna have this boundary here and I'm gonna manage my own data structures and you can only talk to me through this public boundary that I'm gonna define and we're calling that a context so we ship with Ecto by default so by default within a blog directory we're gonna generate a post and a comment schema and these are just data structures that the the blog system uses but if I have some sales system right I can just look at this directory structure and say okay instead of web models blog sales post comment payment order I can have a clear look just by seeing no code at all I can say okay I know this app has a blog system I know it can have posts and comments and I can see okay there's sales system with orders and payments and I know just by looking at this that I can only talk from the sales and the blog system through their API boundaries so the sales system can't just suddenly call into repo insert comment right these things are totally opaque to each other so we're trying to send this message of isolation and I'm going to get a little bit into design patterns here but no one freak out about this but the word context might be a little unusual but as Jose and I were thinking about okay it's like we clearly need to do better about teaching but we have to there's no one size fits all so when we generate code as a best practice we can't we don't know what you're trying to build so the fact that you are passing blog comment there's a ton of information there what do we name the function so it's a balance of we can't assume too much about your app but we want to teach you kind of the best practice to take that code and then grow it into a successful application so we started looking around at prior art and the bounded context pattern for a Martin Fowler is I think kind of resonated because it's not some grand pattern it's pretty much just a set of ideas so this is a little bit opaque here like you know bounded context deals with domainism driven design where different contexts are being explicit about their interrelationships but I like to say basically bounded context make the boundaries of your API is clear and that's the goal so it's not that we're adopting a design pattern from Fowler so I don't start tweeting that like oh Phoenix has now chosen a design pattern it's just we took this we took this concept and we took inspiration from it and it fits well with functional programming because one of the things that happened when I really got into functional programming and Elixir was I started looking like googling for functional design pattern books like I needed to like how do I write my code properly how do I structure it and I thought I was doing it wrong but it turns out like spoiler alert like what you do in functional programming is like you write modules and they have functions and they're well-named and that's it, seriously that's it so just this term bounded context isn't a design pattern it's just you need to think a little bit about the boundaries of your modules and functions and that's the extent of the design that you need if you think properly about that I think you're gonna end up with a successful application so we're taking this idea and like I mentioned with like the sales and the blog system there's another diagram from Fowler where he's just showing that like you have these different boundaries of your app and the sales and support example here like these things can only talk across well-defined APIs so you're not gonna end up in this case of web models where you can just see okay it's like I have a post here and I have an order so then I can just call repo insert and I can kind of cross this boundary I end up with like all these interrelationships and there's no clear separation and again I wanna be able to look at a directory structure of anyone writing an Elixir or a Phoenix project and be able to see exactly what that app's doing and I can say like okay I can just visually form a boundary here and I know that if I wanna talk to these inter modules I need to do it through their public API and then one thing I wanna push that the generators don't do because I think it's not a one-size-fits-all but I want people to start thinking about data-centric schemas if they're using Ecto because Ecto is almost two libraries at this point one is like a data casting and validation library and one's a persistence library so one thing I do in my applications is I model my like I think for most people you wanna keep your view layer concerns out of your core data structures so especially for me like coming from Rails where I'd open up a user model and then I would just have like 30 fields related to like whether the user checked remember me all these like anytime you needed a new feature on some form somewhere it would become a new field it's like these are core data structures of your application let's keep them core to what they're designed for so if I want a registration form in Ecto I can use the embedded schema feature to say okay I can model this form as a data structure pure data structure with the name email password and remember me field and then I can have some registration module that had some well-defined function like register user so instead of like repo insert user right it's going to be no it's like we have a boundary here that says I have a registration that handles user registration and it's going to register a user right there's like intent in that naming and it takes in some a raw map of data it casts it as an embedded schema which just returns a accounts registration data structure and then internally kind of my last step is I convert that to my core data structure and I persisted in whatever means possible so this is what this looks like and again we don't generate this by default but this is what I want people to start thinking about is using Ecto's multi features I can take in a registration change set which is just an embedded schema so I'm not connected to the database and I can apply the changes to it to see if it's valid so that I'm just casting user input into a raw data structure if that data structure happens to be valid the last step I'm going to do is take that data convert it into a user and then insert that user into the database so a little bit more code than you have to write but in this way I'm not polluting my account user with whatever feature I need on some form somewhere right I'm keeping it isolated to parts of the application that require it so this is just the kind of things I want people to start thinking about when they're designing any kind of application that touches the database especially with Ecto so part of this isolation message is getting people to focus on okay it's like we're isolating our modules we're isolating the boundaries of our APIs so maybe we should isolate our applications so Phoenix will ship with a dash dash umbrella flag to generate an umbrella project like mixed new umbrella and it's just not I don't think every project needs to be an umbrella but I do think umbrella projects are kind of underserved in the community so we're trying to walk through the merits of them why you wouldn't use them but if you pass the umbrella flag in Phoenix you can notice we want to continue to send the same message so instead of live and then a web directory you pass dash dash umbrella to mix Phoenix new we're going to generate an app that's your application domain that's where your app lives right and then we're going to generate a web app with like whatever if you type in my app we're going to make it my app web and that's the web interface and then these are literally isolated like the only way they can talk to each other is across the boundary of the applications so your app would not actually depend on the web application but the web application and its own mixed dependencies in the umbrella would say okay I'm going to depend on this app so we can continue to send this message of literally you have a web interface here to some greater application domain so the whole point of Phoenix 1.3 is to get you to think about boundaries so it's not the that we can design your whole application but you're going to make you think just a little bit upfront about your design so instead of like when you're scaffolding an application you want to generate something some crud-based it's easy just to say run a few commands and persist the data and it mostly works how you want it but then a year down the road two years down the road now you've tried to grow that code and it's an abomination so we want to think we're going to make you think a little bit upfront a little bit of effort to think okay like where do things live how should this be named and we think that this hopefully is going to pay dividends for you and the rest of the community in the future so I'm going to take you on a little bit of a thought experiment based on adding a feature to an application and we can kind of see where where we're going with these ideas so let's say that we want to add some reaction feature for liking posts right am I going to like a post am I going to dislike a post or you know Facebook has like the rage face whatever prior to Phoenix 1.2 the kind of mode of operation would be it's like well I have a web models directory and I know I want to put this in Postgres so I'm just going to add a reaction ex file and now I have three models here right but the problem is we're not you're not thinking about the boundaries here you're not thinking how this relates to the application and there's no we're not giving you indication that you should be thinking any other way right you'd have to have some intuition on the design here so we know that this isn't necessarily where we want to push people and it's worse because in your controller the way we were pushing you is you'd say okay I have this code here and this is where Phoenix wrote my code so this must be where I write my code and that you would start having to put ASOC here right I have to associate multiple things and have the controller be concerned with not only how the data is inserted but how do I actually associate it so we know that this is not necessarily what we want but now imagine you're using Phoenix 1.3 and you've used the generators your directory structure looks like this so now you come to this directory structure and you're like okay I'm gonna add a reaction feature by default even if you're using the generators to generate the reaction feature you have to think it's like hmm where does this live there's no dumping ground anymore right we can't we won't give you the easy out so you have to it's like okay well where do I where do I save this file so you can think by default like well I want to be able to like post in comments so maybe I add it here right and maybe this is the right answer but you have to think a little bit up front so at least you're putting it here so let's let's run through this let's say okay this is part of the blog system I can only have posts and comments that I can like it should live here and then I can compose something really nicely like this this Phoenix is gonna generate a function for me to kind of give me this hint I can say okay get the post and then pass the post into the blog system let's like it nice and pretty the controller doesn't have to know anything about it and then within the blog module this is where we've written your code for you before if you're not even really thinking about greater design right your newcomer you just want to get started you at least see like oh Phoenix generated this blog module this is where it's doing the persistence and this must be where I write my code so even if this is like the minimal jump that you can make you'd be like oh this is where I write my code there's a module here in functions then we've won so at minimum we want people to get here where they're like okay I'm gonna add a function like post and this is where I'm gonna do the associations and I'm gonna do the database persistence or whatever I need to do maybe this stores it in an agent or ads but if we get people here we actually have a well-defined API that has reusable code that you can grow in the future right if like a liking a post and now something we want to cash into ads we can do it right we don't have to change our web interface or any other any of the other call sites but my hope is we get people to think a little bit up front so I'm not saying the previous way was bad but if you're thinking about boundaries you're gonna say okay I'm gonna add this new feature you open up lib and you're like hmm I've got a blog system I've got posts and comments and you're like you know what a reaction system itself there's a boundary there right what if I I'm gonna later add new resources into my system I could add user profiles that people could like videos anything it's like maybe I should have this as its own boundary with its own data structures and own well-defined API so our hope is people think okay I can create a new module and functions to do this thing here and then in my controller I can actually have these things completely isolated the controller can just say okay if you want to like a post then we can actually call and we have a module for that get the post and then reaction add like right and these things don't even have to know about each other necessarily so this could be nice because we don't have to depend on the blog and reaction modules don't have to depend on one another but you may also say like well I don't want to have to call into this explicitly to get the light count every time maybe every time I fetch a post or a list of posts maybe I want the likes well then the blog module can depend on the reaction module and they could just call a function right but the whole goal is these things can only communicate across their well-defined boundaries so the fact that they could both be using Ecto and persisting that data is transparent to the modules right there's only a well-defined boundary and if I want to add a like the blog module can just talk to the reaction module through that boundary and then I can put the likes into the posts and now I have a nice abstracted reacting system right and I know Phoenix is pushing me towards making this boundary so I can think okay my API is gonna look something like this add like remove like get the light counts this is nice as succinct so now the jump that we think people might make is okay I went from adding this file into a dumping ground to thinking about maybe it can live in the blog system to thinking about oh maybe it's its own boundary as a module but now you're like well if it manages its own storage right it has its own boundary that can like any kind of resource what if I had an umbrella app and I could take the leap and say you know what if this thing is totally isolated instead of being isolated module maybe it's an entire isolated application so this is just kind of the way we want people to think so I'm not saying go home and rip every feature out as a module or just make everything an application so don't read too much into this but at least think a little bit about your intent right and we want people to start being able to make these leaps earlier on and if this is an isolated system so if this reaction system manages its own storage it can be starting to stop deployed in isolation and then it could be put on hex and shared right so if someone makes a reaction engine right they could put that on a hex and now anyone that wants to be able to have resources in their app liked, hated, disliked I can just drop that as in as a dependency maybe configure it with a couple lines of code and now we've shared this feature across the entire community so these are the kind of things I want people to start thinking about so don't go home and start ripping out all your code but just start thinking a little bit about the kind of intentions of your APIs and the boundaries of your modules but naturally it comes up like should or shouldn't you use an umbrella and I was thinking like really hard about this because like at Dockyard I'd say probably 50% of our projects are umbrellas and I think there's a push that everyone thinks like oh umbrellas it's a new secret sauce so everything should be in an umbrella and I don't think that's correct so I started out like okay I'm gonna make this flow chart it's gonna be like it's gonna be really impressive but it turns out like for me personally like this is pretty much my thought process though this didn't need to be a flow chart but I was excited about making a flow chart and this is what I ended up with but I bowed it down to this because this is a hard problem right it's like we don't wanna over optimize too early on but it came down for me that if my context so my module manages its own storage if the answer is it doesn't I'm using some shared database in my application some shared storage engine then I keep it in the same app because for me those things are necessarily coupled so I see some people that like they split their application a bunch of different umbrella applications that all depend on the same repo and some other application that's depending on Postgres it's like those things are now have like implicit dependencies across each other that like if you deploy one app and you need to run a migration like it's just you lose site operationally what's going on so for me if unless it manages its own storage I don't even think about splitting it out but if it can be isolated and it is using its own et's table it is separate from my active Postgres database and I at least consider an umbrella so I don't say like immediately extract it but at least it's worth considering like this thing is totally isolated if it manages how its data is stored whether it's in a process whether it's in another database anything then it can be deployed in isolation starting to stop in isolation and it's worth considering extracting it out so I've kind of you know I've been hyping like extract extract isolate isolate because this is what I want people to think but I also want to then put the brakes on a little bit that like if everyone goes home and gets too carried away that you know everything everything taking and like taking step by step to extract like it feels like you're making your code better but it can actually increase the complexity of your application so anytime you decouple your data model so like you know databases get a lot get a bad rap but anytime you try to decouple your data so like you know what we don't need to put this in Postgres let's just put it in a process right maybe it's ephemeralish data that doesn't necessarily need to stay around forever but every time you decouple your data model you then have to increase the complexity of your app because anytime you want to join or aggregate data you're now have to stitch that together by hand so it's something that you have to think about it may be worth it and maybe not so here's one example so this blog application so a trivial feature right I if I could store this data in Ecto and then if I want to get the light counts from my posts I could say repo preload aggregate that data in one line of code with Ecto and I have all the light counts but if I decided to write this reaction engine now that is isolated manages its own storage has a API boundary there I have to write code like this where I had to say okay well now if I want the blog every time I get a blog post if I want that light count value I need to call into reaction I need to have an API that takes a list of post IDs and then returns all the likes for any of those IDs and then go through each of those and if there wasn't like for it then put the likes into that data structure like it's not terrible right this is a small amount of code but every time you make one of these decisions to isolate components it's now on you to aggregate and stitch the data together so just think about isolation design with intent but then think carefully about the trade-offs right because it's not always the best idea so that's the end of my talk Phoenix 1.3 is like I said making you think a little bit but I think in the future we go one years, two years down the road or I inherit applications as a consultancy and I want to hope that we open up a directory structure and see some kind of like intentional design instead of web models, 100 files we want to see the feature set of the app and I think as a community if we do that we'll be able to generate new features that are shareable that are taking the best of the ecosystem as an offer right so like I said don't get too carried away put the caution on but design with intent so that's all I have all right so I did .NET and DDD for probably 10 years and I was looking at this bounded context concept and I'm curious like how far down that DDD rabbit hole you went and if you did go fairly deep did you look at the aggregate and aggregate root concepts yeah so I looked at quite a bit but the problem is like I was also interested in like a CQRS design you know I I've done a lot of looking so this Phoenix 1.3 was actually it was like I promised it was out by the end of the year in Orlando last year and it's taken a lot longer because the code hasn't been hard but trying to make like trying like this is these are best practices so it's like how much do we put on people that is the right design for a broad use case right because it's like anytime you apply any kind of decision in your application it's like it's going to vary on your it's going to vary on your use case so for us this is just getting people to think about isolated modules and functions with well-named meanings right and then if you want to use aggregates right if you want to apply that then that's great but we don't want to tell people like the last thing I wanted was to have like okay Phoenix now uses like I look in the CQRS because like personally it was exciting but then I was like okay Phoenix 1.3 is shipped now read this 400 page book so I think that I'm very interested in those ideas and how we can solve like how we can better aggregate data so if people do have those questions now it's like okay I followed this idea I have this isolated component now how can I make this code better that's definitely a great idea but for the default someone two weeks into the ecosystem it'd be too much Hey Chris I'm just interested to know if you were studying Phoenix again today thinking about these concepts how differently would it be structured? How would the lib be structured? It's Phoenix So I think how we're doing it now but so yeah it's tricky because like there's always there's a backstory there because initially web there was no web directory at the root level so this is where like that was actually Jose pushed me in that direction which is kind of funny too because people that complained about it in the past are like oh Phoenix is special because this web directory I'm like Jose himself he was the one I mean I agree but yeah so I think a lot of this is a learning process for Jose and I as well about these pushing people in these right directions and it's always a balance of like you know how much do you push and how much like because every time we push we potentially put up a barrier because at the end of the day someone is coming into the ecosystem and they're like I just want to I just want to insert a post in the database like I don't like you know what I mean it's a balance so I think we're at a happy balance here you know obviously this is how I would design it you know if I can go back that's how I would design it but talk to me in three years and maybe the answer would be different I'm going to talk to you next year maybe I just want to say you're talking here about how you how you design the user experience and partition that up I'm talking about how you design the framework itself like the code of the framework itself so I don't think the code of the framework itself would change much it depends on that because Phoenix the the stateful bits the interesting bits for me are in Phoenix Pub sub now so Phoenix core is mostly a plug extraction and you know about transforming a connection and building a feature set around that rendering you know taking that data and rendering it in the Phoenix view layer so I think like our neat features in Phoenix core are mostly transformational features so like taking the data pre-compiling the template Phoenix Pub sub definitely was more of a learning process where my initial take at Phoenix Pub sub was a disaster it was horrible that's where like Jose was like wow that's you know basically nicely said this this could be a lot better but the high level API that I got was was what I wanted so like our channel layer worked exactly how I wanted it but I didn't know enough of OTP in how it should be structuring the underlying bits so that would be totally different but I don't think it'd be it's not something that I think I could have done any better at the time something I had to kind of level up into so as Phoenix evolves and we're making these changes on how you think about how to organize your structure we've seen two over the past three or four versions had several distinct changes do we have a web directory now we don't have a web directory now we're having these contexts are you worried at all about sort of this semi whiplash of hurting adoption and getting people confused as they're either looking at documentation and a new version comes out and all of a sudden they have to switch their mental thinking or concerns like that yeah I mean it's definitely a big concern because the like one and then like the guides have to be rewritten so I had planned on releasing one three with the guides done and I'm going to release one three this month it this month it's happening it's almost done but then the guides are going to be my next effort and the Phoenix book has to be you know has to have a new edition out so it's just personally it's a big undertaking but then also like that's why it's taken so long to fill out these ideas like the data-centric schemas I mentioned that was I wrote the code generators to actually do that ecto-multi embedded schemas by default and then ended up back trapping because it was just too much it wasn't a ton of code it's code that I would write personally but you have to balance that getting started experience and as far as the changes in whiplash I think this I think this change will not be this is not gonna be whiplashy I'll give you a reason because all we're doing is regenerating modules and functions right we're not there's no when you generate a context there's no use Phoenix context it's literally def whatever and a function definition since we use ecto by default it's gonna have your repo insert code so I think there's no backlash because that's how I think that's how if you're in the ecosystem and you're an advanced programmer you're gonna say okay I have this web interface if I'm not thinking about bounding context I have some feature I'm gonna write a module I'm gonna write functions in it so I don't see some giant course correction from here like I don't think it's gonna be like oh I should have we have to go back and write all our code in the controllers now so I don't think that there'll be a big course correction but it is something I have to weigh heavily with new ideas right like if we did introduce like I wanna I'm interested in service discovery right so if I wanna introduce these discoverable services and processes every time I do that I have to think like this has gotta be a stepping stone to kind of my future goals. So Chris I really wanna commend you on how much you are thinking about bringing the new users along I think it's a big deal that you're up here saying that scaffolding isn't the way we're going to be writing applications it's the way that we teach and we need a mature book we need the documents all to be in line and that patience I think does a great justification or does great things for the community. Thank you. So your flowchart where you're deciding whether to break into an umbrella application is pretty similar to our flowchart when we might split something into its own get hub repo and put it up as a microservice so I'm wondering like do you have any opinions on when you might choose one approach over the other like what might go into that decision? Sure yes so if it's an elixir so if we're strictly speaking as like all those microservices are elixir applications and they're all part of the same greater platform that you're writing I would just put it all in one umbrella app and granted it's like depending on your team size and the commits and conflicts with merging but for me umbrellas give us the microservice architecture like we can define so here's one example that I didn't mention like at Dockyard we made it by default I chose an umbrella for this application and it was a pretty simple application but it had its main purpose was to connect to Amazon SQS queue that was powered by other parts of their domain or other applications not in elixir and it was gonna serve some channel interface on the browser so ahead of time I said okay we've got this staple thing that's gonna be listening on a queue and this thing can be that thing could be deployed in isolation so if you think about microservices like I can have 10 web front ends because I want to load balance the web front end but I want to consume this queue I don't want to have every web front end I deploy shouldn't be running a queue consumer I only need one of these things processing messages and deciding how to you know broadcast them so for me it's gonna depend on your deployments so the micro all the benefits of microservices through deployments starting and stopping in isolation you get from an umbrella app so the only reason I would even think about splitting as a separate get repo is if I had like a huge team who couldn't manage a big code base but I think isn't like Facebook's I think Facebook's whole code base is like one repo but anyway yeah so I think there's no benefit in splitting I think there's a huge benefit in keeping everything underneath one if it's all in elixir because then if you can deploy these applications together and you can cluster them what used to be you know RabbitMQ or HTTP JSON API calls now just a gen server call internally so right if I have some name process from one of my apps I can just say gen server call whatever and now I'm talking to this app across the cluster so it's a huge one for me to keep them together I think the million dollar question is do you have a pulse on an ETA and then also when you do have an existing model folder that has tons of models and just a dumping ground is it possible right now to basically take those and start organizing or not possible is it recommended approach to start organizing those in a more bounded context to move those later to 1.3? Yeah I mean you can move them now so like 1.3 as far as like we've got some minor features so action fallback is probably the bigger feature there's just you know a handful of APIs like being able to get your current URL in the controller so mostly it's about code organization and the majority of the new code from the commits is just into the generators so if you're not using generators you can just put your code into the new format so I would recommend that I think it'd be interesting I think for me to talk to people who have gone down that path because I think you're probably gonna find like me like when we were writing the Phoenix book that we ran into these cases where we were like wow if we would have thought about isolation just a little bit upfront we wouldn't have had to end up refactoring our code later so I think as you start trying to isolate these pieces there's gotta be spots in the code that you're like wow there's really no reason I should have coupled there but since you didn't have to think about it not even as a beginner it's just as an expert programmer like if you're not having to think a little bit that could, it's gonna change your design so I think yeah start splitting now and let me know how it goes oh ETA so I said this month so the generators are all in place now so on the flight here I polished the documentation at this point I just need to add a few checks like if you try to run the generators in an umbrella now like if you try to run Mixed Phoenix Gen blog comment from an umbrella route it generates the files like inside lib in the umbrella route so there's just a few like validation checks and then I need to walk through Josie and I need to kind of go through with the fine tooth comb and just make sure like the conventions that we want and the generated code are like fully baked so this month it's gonna happen. In response to the last question my talk at 330 is exactly about you had a monolith you broke it up and then we broke it up into an umbrella yeah shatter it into an umbrella yeah so to take Luke's and Luke are you you have to redo a bunch of slides or are you good? But no major scramble to okay yeah because like I said at the end of the day these ideas aren't like let's go down this uncanny valley of like new design patterns new ideas like the big idea is like oh we can have modules and functions that have well-defined clear APIs and I think like no matter what everyone can agree that that's a good thing and that's kind of the starting point that we're pushing people towards and how you grow from there is gonna depend on on your you know your application. I really like what you just talked about I think it makes sense I think you should always try to do the right thing you know even though it's gonna have some collateral damage and so on I think you know it can be avoided but if you keep doing the right thing I think you're on a good purpose. One thing that I see you know in applications it's a lot of time in the model level there's a lot of business logic you know how is the data going to be inserted how's the data going to be queried and that's where the kitchen sinks you know syndrome comes into right what are your thoughts about you know how about just treating storage just for what it is you know I've got insert you know correct type stuff, validation but where do you put the business logic you know does that go in a separate umbrella you know those rules about how people are actually gonna use the app leaving their own content somewhere or what are your thoughts on that? Yeah so for me they're necessarily coupled together not always but I think like if I call into let's see if I go back here like the whole goal with like blog list comments right by default I don't know if I generate that or not but by default all list comments is gonna do is call repo all comment right but the caller doesn't know that we're forming an actor repo and we're hitting a push for database right so to me that's not necessarily the details of that boundary and you can split like it's not that we need some like God module there in blog like that blog module can split out its logic but for me personally storage the storage of my app usually lives alongside the business logic lives alongside how that storage is handled but the callers don't know necessarily the details of that right and I'll add one more thing though so one more thing so part of this is like if we talk about like peer functions so if I'm writing some authorization layer in my app where is Nico in here from Dockyard? Yeah Nico's in the back I think Nico wrote a blog post for Dockyard like how do you to write some authorization layer like can a user like a policy type system like do I have authorization to take certain actions on certain resources you know that isn't for me so that's quite a bit of business logic there but it's all pure functions so for me that would be one case where we're gonna define a module with function definitions and they just operate they operate on data structures they don't have to fetch any data they get a user they get a post and they get the users you know they have the data that they need to make the decision so I think in general it's best to structure your code is pure functions where possible but I wouldn't then split that out in a separate app or I don't see the value of splitting that out in a separate app. I mean I think it kind of goes a long way in terms of still trying to figure out what is the intent of the application when I define application actually tend to have a store umbrella type project and also business logic which is kind of a common ground where people can go and look at what is this application actually doing so maybe another point of reference I would think in the application. Yep. But another comment I wasn't, I'm not sure what it's called now but when you had the action action callback or action fallback rather. So there are a few lovely and comfortable maybe you know it's my go background so in go you know when you make a call you know you have the OK comma signature right? Here I think you know there's a little bit of magic you know I think if there is I agree there's repetition about every time having to check for the return signature but I think it clarifies what the intent you know this thing can fail you know and I think being kind of maybe less magic but yeah maybe a little bit of redundancy you know in terms of checking still clarifies what the intent of the call is my just my opinion. Yeah so in this case so we thought like I think if you watch my Orlando talk I called this a responder which is kind of a troll on the Rails community but it's it's a similar so we didn't want to introduce this like new concept right like the whole the goal was that we had these data structures that we handled in the same way every time yeah and it's like what do we call this thing like you know it's a whole job is to send a response like so Jay-Z and I spent like this is where like this is why stuff just takes so long it's like you know I've had at thesaurus.com up like what do I call this thing it's like we were like oh we just call it like a handler it's like you know handlers like if there's like anytime you name something a handler it's like a cop out like there's like there's a cowboy handler in the and there's like a few handlers in the Phoenix code base so like it's like we can't call it a handler like so anyway so that's like this may look like magic but the goal is like you have this plug pipe one and the plug contract is you take a connection in you return a connection and anytime you don't do that it's gonna fail all right it's gonna raise an exception rightly so so we said okay like the whole point of this thing is really just to take a value and turn it into a connection and that's all it does and action fallback while that is a macro all it is is a plug so it's like it was actually beautiful that Jose and I were like you know let's introduce this responder idea and then that can you could say use Phoenix responder that can define some functions and we pass them in and then Jose was like you know what this is almost like just like a two con function like you know takes a value and converts it to a connection and we were like oh wait we have a contract for that like that's the plug contract it's a def call and takes a connection and a value as second argument so like well we have the connection as the value passed in before the action and we need that data if we don't get the connection back so we actually had this beautiful contract already in there which is plugged so there's the only magic here is that we wrap your controller functions and say if you return something that's not a connection then we just call this plug and that's it so it's very very little magic all right let's give it up again