 I actually changed the name of my talk. So if you were coming and expecting a big brand about Rails, sorry. I was going to do that. I was going to be really cathartic. I was going to feel great but as I was preparing the talk I decided that I actually wanted to be a little bit more constructive. I didn't just want to brand about Rails. I wanted to provide some suggestions for how Rails would be better. But so I was trying to think, so I have all these ideas. As I was trying to pull apart this table of ideas, I came across this common theme and this was of sustainable productivity. This actually fits really well into the talks we've just seen from Stephen and Mike. I'm hoping to watch from where they started and go into some of my ideas on it. So this idea of sustainable productivity, or these words, actually stole directly out of the Rails self-descriptive. If you go to the Rails homepage, this is the description that you get. And the reason I picked on these two words is because I don't actually think this is something that Rails can claim yet. I don't believe that Rails is optimized for sustainable productivity. And that's sort of the hypothesis I'm going to be presenting in this talk. So I need to make a distinction between the types of applications. So prototypes, things we need to get out of the door very, very quickly. And what I say, hopefully your application, applications that, this is the sort of bigger application, this is the one-year, three-year, five-year application. It's got many developers. And where we're starting to get a bit stuck as far as our productivity is concerned. This is the type of application that I'm concerned about in this talk. The prototyping in a bit is very, very important, but is not my concern. To do this, I want to make another distinction between easy and simple. This is a distinction that was first introduced to me in Rich Higgie's talk at Strangelove last year. Simple made easy. It's one of those rare talks that sticks in my brain and never leaves. I highly recommend you watch it if it happens. I'm not going to do his topics, his concepts of justice here, but the quick summary is that simple things are things that do one thing well and they're not interleaved with one another. And what's interesting about this definition is that it's kind of objective. Like you can look at a bit of code and say there's this bit of code and there's this bit of code and they're either intertwined with one another or they're not. In contrast, there are easy things. Easy things. Rich sort of defines easiest things that are near to hand. Things that we already know how to do. Things that we can reach to and get things out the door very, very quickly. But they're not necessarily simple. Easy things can in fact become within. And my hypothesis is that Rails is easy, but it's not simple. It doesn't have to be in this way. As I said, this is a constructive talk. This isn't me just picking on Rails. This is these things that I generally think that we could do better. And this is both directed at the Rails framework, but also the Rails community. Things that the framework has nothing to do with, but as a community, I think we could do better to focus on the sustainable productivity element. I've shaped the talk into three sort of major topics. They're all kind of interrelated, but I wanted to fence post it somewhat. So I'm going to be talking about data. I'm going to be talking about coding in small and I'm going to be talking about coding in large. But we'll start with data. And specifically, we'll begin with how we store our data and databases. This is the segment where I'm most critical of Rails as a framework. This here is the hypothesis that I'm putting forward. It's not just that Rails doesn't quite have the tools to do what we need. It's that it is actively discouraging the tools that we should be using. And I think this is the real problem, which is why I'm bringing it up first. So let's start with data integrity as something to focus on. Now, data integrity problems, there's the obvious data integrity problem, right, where you have corrupt data or you're missing data for a user or things that are obviously bad and causing breakage. I'm not really interested in talking about that aspect of it. I think that's fairly unpromptual, because that's bad. I'm interested in two other aspects. One, that data that lacks integrity, you need more code to deal with it. You need extra nil checks, extra sanity checks in your code. You need to increase the complexity of your code to deal with data that lacks integrity. Further, data that lacks integrity is harder to understand. If you're an engineer and you come across some data which lacks integrity, it just doesn't make sense. And this is stuff that you have to spend your time trying to figure out how it works rather than actually being productive. So a lack of data integrity kills sustainable productivity. So if you've been doing Rails programming for a while, this is a fairly cliched example, but to quickly go over it for the new people in the room, this is a classic example of if you're writing a web-style application where you have, by definition, you have the transparency to deal with, pretty much. We have this issue where we can put a validator, you need this constraint on our username. We can have one process, sorry, one request come in and be handled by one process. We have another request for the same action. And with another process, they both ask the database, hey, do you have a record with name Xavier? The database says, no, not yet. So then they both write to the database, you end up with duplicate data, which is now violating your validation. We have invalid data in our database, despite the fact that we told Rails that we didn't want to have this. So this is like a fundamental class of problem. This isn't something that just happens, like sometimes it's like theoretically possible that doesn't happen. This is a fundamental problem in web applications. And as such, we need to have a way of dealing with this. Every project here is going to have to deal with this. So we don't have to deal with this one, we can have a unique index. Rails tells us this is in the documentation, this is great. We put a unique index on the database table. But then how do we deal with that? The first five lines of this method should look familiar to you. It's how we write Rails controllers. But now we have this extra kind of validation that we have to deal with. What happens is when the database validation actually throws us an exception. There's actually a number of good answers for what goes here. But my concern is, we don't have a best practice for it. This code looks weird. And what this means is that as a developer, when you come across this situation, you have to figure out what to do. Therefore, and if you're figuring out what to do, you're not being productive on other areas. So this is one aspect of how Rails doesn't provide us with the necessary best practices to deal with that. So I just showed you this example. And this is one example which is in the Rails documentation that says, hey, if you need an index. But there's other classes, like this same class problem pops up throughout the Rails framework. And we don't talk about it. We don't deal with this properly. We have a has one association, exactly the same problem. You can get a has one association that has two child records. This is confusing. You need a unique index. But in my experience, people don't put unique indexes on has ones. As many, same problem. You have one process creating a child record. You have one process deleting the parent. You have a data integrity problem. You have a child that doesn't have a parent. But Rails has no way to deal with this. The easy solution is to use a warranty, which most other frameworks and people who have done database stuff know about. But as far as Rails doesn't exist. I'm not suggesting that we go back to putting everything into our database. I don't want store procedures. I don't want, you know, development done inside the database. But these examples that I've shown, they're at a high risk of data corruption. What I mean by that is, say you have an email address and it's found its way into your database. There's not a high risk that an email address is going to be corrupted. We've validated it once. There's not really a good way that you can violate the integrity of that specific case of data. It's come through your Rails validation which is great. Whereas these other things where we have Jupyter data, we have data missing parents, you can't deal with at the Rails application server level. Well, sorry, you can, but database constraints are the best way to deal with it. So this is something that four way that applications we have to deal with. Database constraints are the best tool to do this. And Rails doesn't support these database constraints, or provides very, very primitive support, I should say. But what this is in effect saying is that Rails doesn't support the simplest tool for the job. It's something that is fundamental to the applications that we build. This is embarrassing, I think. And online projects, it can totally kill our productivity. We just don't have the support around it. To back that statement up, here's just a small list of ways in which Rails does not support these database constraints. The conventions I've spoken about, fixtures, if you use them, they don't get added in the right manner, it allows you to use foreign keys. The Ruby schema format just totally ignores your foreign keys and key constraints. If you switch away from the Ruby schema to the SQL schema, and you're using my SQL, it still doesn't work, it still doesn't dump your foreign keys. Though that may have been fixed in 3.1, something, or 3.2. There wasn't quite the same. But anyway, it's only a very recent fix. Migrations, I teach, I have taught a lot of people how to use Rails, can be part of the frameworks. And when we run a migration, we have the T.Vol references. And people just assume that this has a foreign key. And I'm like, no, it just means reference is used and you should use your ID. And people don't believe you, they're like, that's stupid. Anyway, migration is also, I didn't talk about null constraints, but by default, it makes all your columns nullable. Which is a terrible idea. As a default, null doesn't make any sense. Neither does that sentence, whatever. Point of relationship, so we also, Rails provides us with techniques that are easy in that, hey, we can model some data, but they're not simple. And they don't assist us to preserve our data and keep it. Our relationships, I'm going to skip over very, very quickly, because I don't quite have enough time to cover everything. But in brief, here's what they do, if you don't know about, we'll move them up. There's other ways of doing it. We can duplicate our tables, rather than having one global comment table. This isn't a violation of drive. We can do this in code. Having separate tables, we can use mixins and loops, so we don't have to duplicate code, but we can have a sane and data structure. If you really have to, there's another way of doing it, where you sort of use a class table inheritance type thing. It's pretty heavy weight, but if you really, really need to aggregate overall comments, it can help. Or you can use more of a document, either like HNOR, Postgres, or you know, any one of the number of documents. There's another, my point being that there are a number of simpler options that may not necessarily be easy, and that they're not near-adherent, but over the life of a project, will greatly contribute to your productivity. From that, I want to segue into talking about some smaller bits of code. Now, in this section, I'm not really talking about the Rails framework at all. I'm just talking about some techniques that I've found not to be not particularly common, and that I would like to see more of. And the first one is that when we talk about data, it's not just data in our database. It's also how we represent data in our code. And to do this, I'm going to show some code. So, this is a check. It looks at a file for a number, and checks that it's greater than 90. I use this in a tool that I'll show later on. The point. And here's a first cut of, you know, how you might write this code. We read a value out of a file. Say, we read our coverage at 89%. We checked that it's greater than 90. If not, we created a violation to say, hey, you didn't meet your coverage quota. If we can't read the file, we create this other violation. Now, this code is already changed, but there's two problems. One, there's a bit of duplication. Kind of in the violation.new. It's not so bad. The actual problem with this code is it's complex. In the definition I used at the start, it's taken two things, reading out of a file and doing some business logic and intertwine them. And taken, like, in isolation, this isn't a huge deal, but summing up all these interleavings across the scope of your project can be a really big problem. So, here's how you would separate these two things. Here's how you would have two simple components that can work together without directly interleaving. And what's interesting about this is in order to separate and make this code simpler, I've added a new concept, that of the unavailable value. This is counter intuitive. We are adding more code, but we've made the code simpler. Not only that, we now have a way of talking about this concept, which in this example is kind of implicit. We still have an unavailable value here, but we have to infer it out of the code. By moving to this model, it becomes more explicit. And it gives us something to talk about, and it gives us a place to document the thing that we're talking about. The word for this is called ratifier revocation. It's making something abstract, concrete or real. I've just given you another thing that we can now talk about and use. I had another example of what I was going to do. So, what I've shown there is I showed that we had this place, this thing that we can now talk about. I also showed that we now have a natural place to document this thing. And documentation at the class level is very, very important, and we don't see enough of it. And this type of class documentation can't be replaced by self-documenting code. I think there's been this shift towards we write code that has really long method names that describe what the method does and they're intent revealing. And this is excellent. We need this, but it doesn't give us all the information we need to know about the code base. Here's an example. I was on-call. My first week on-call at-square. And we had a background job fail. It was this job. And so I'm like, all right, let's find out what this does. So I open up the job. And this code is pretty much self-documenting. As in, you look at it, serializes the reader request, submits an order, it plays a shipping batch, right? But what do I do now? I've got this failed job. Do we care? Is there a deadline? What's shifting? Like, there's all these questions about this code, which are actually more important than the code itself. And this is the sort of problem you run into as you get bigger code bases. More developers. I had to spend the morning running around talking to people, trying to figure out, first, is this a big deal and do I need to fix it now? And, two, how do I fix it? Like, what do I do? Anyway, I did all that and I documented it all. And it shows you, you know, how this fits into the context of a bigger business process. You know, who's shipped team are, that we have a deadline. And, importantly, at the bottom, I can just safely rewrite this code. Now, this sort of thing is vitally important. It doesn't have to be a new code mix, but a link to it should at least be at the entry point to your code, meaning that most of the times I'm going to come into the code via an exception or an error or something like that. This is the first thing I'm going to open up. I shouldn't have to go hunting around on a wiki page somewhere in a closet with a sign saying, you might have caught it. Or you might have seen this. This is taken off. The best, we know this is useful because I did this. And then next time I was on call, without saying anything, 70% of our jobs had documentation like this. People found it useful. And I think, to me, that's the most telling form of success in that it's something just patches on because it's useful for people. I think that's really powerful. So that's documentation. Consistency is also a form of documentation. Staying with background jobs for a while. Here's how this is an auto-retry background job. Now whether or not you really agree with the style or whatever, it's a reasonably decent way of doing job retry. This is fine. This is great. Problem is, we have another job, which has another decently fine way of retrying jobs. But it's got a different retry time. It's got a different retry count. Now if I'm adding a new job, how do I put retry logic in? Which one do I copy? Do I copy? Are these right? How are these specced? They both specced differently. I pretty much have to reinvent this retry logic now as a developer putting in some new code. And if I'm reinventing retry logic or trying to figure out which code I should use, I'm not being productive on other pages. So consistency is more and more important. The bigger and more developers you have. The bigger code base and more developers you have. Because trying to interpret inconsistency is not developing pictures. We solve that by having a mix-in, retribal job. Every job does this now. And the extra benefit we get is that the specs are really easy. So if we have one implementation, the problem when we have four implementations of retribal jobs is that one is going to be buggy. And this was true in our case. One of them had a buggy. And if we have done once and done well, it's a little bit more effort, but it's worth it for this sustained retry theory. The last point I want to talk about as far as coding in small is concerned is error rates. The more developers you have, you can be a very, very good developer with only a 1% error rate. If you have 10 developers, your error rates are down to 90%. And this is a multiply-overland-view project. Error isn't really the right word, I think. When I say error, people think, oh, we can fix that. Then we get fixed before we go to production. A better word perhaps is stupidity. And I don't mean having a derogatory sense, but I mean it in the things where, if you write some code and then I came up to you the day after and said, dude, what's this code? And you're like, oh, did I really commit that? Like this is the stuff where, when quiz on it, you wouldn't commit. But this happens. Like maybe you haven't had your coffee for the day. Maybe you were up late last night. Maybe you're thinking about a relationship. Maybe any number of the things, you are not going to be on your game 40 hours plus a week. It's just not possible. Especially not possible across 10 or more people. So we've got ways to fix this. We've got pairing, there's one. Code of view is also a great way of catching this stupidity. But honestly the cheapest way to catch this stuff is with machines. There's a good class of problem for which we can pick up really, really easily with code. And we should. We should be failing our builds for this stuff. If somebody checks in some code with some dodgy white space, failure build. The reason is, because if you don't do it with a build, it means when somebody's doing a code view, they'll be like, oh, your spacing's out, man. And they're not, even if I don't tell you, they think about it subconsciously. The style doesn't match up. They're not thinking about logic. They're thinking about style issues. Catch this stuff as easily as you can. Now, style check is kind of a bad rap. You need to be careful that you don't go too far with this. There's certain types of style where, like, the code throws up an error and you're like, oh no, actually it's all wrong. I'm right. Let's throw this thing out. So you need to be very cautious about the things that you do check. But good news is, there's a gem that is cautious. It has really good success rates. And it's brilliant and I approach it. So anyway, there's this gem called Cain. It does this. If you don't like any of my choices, you can turn them all off. This is the cheapest way to deal with this problem. You can run it before you commit. It's really easy. I use this on projects I do by myself and it attaches problems. I find this very useful for personal things. It'll say, hey, do you know that you're about to check in the most complicated method on the project? Do you really want to do that? Is that something? Just have a think about that first. That is really useful. I'll go one step further. If the tool is good enough, if it has a high enough accuracy rate, you should comport the tool in the one percent of times where you disagree with. The reason being that having this baseline of, you know, the tool says zero, it's good. If you can try to pull it off, it's worth the compromise, especially on a team, to be able to check this stuff very quickly and at low cost. That's somewhat controversial, but suck it up and do it because it's really useful. Anyway, current the large. Some guys have been talking about this already. Mike used the word monolithic as the number one problem. I think the core problem is actually dependencies or interrelated dependencies that we're not really aware of. The monolithic app isn't really bad unless all the dependencies are all over the place. In every model that you have I've seen, that's the text, so it's kind of an accurate argument, but dependency is a really big problem. You read any book on software development. This is what they talk about. We're talking about Eucone, CTM, they all talk about dependency. Let's talk about the dependency between our app and other things. This is the most obvious form of dependency. The most important way to deal with this is that you frame the dependency in your own terms. We're going to explain this with an example. Update method, sending an email. The problem is using rescue, so we're doing, we're incurring a job. Problems, if we do this everywhere, this is what our dependency tree looks like. We have all these bits actually depending on rescue. The thing is we've tied ourselves not only to rescue, but to this idea of including background jobs. That's not really what, we don't have the intention of really mentioning here. And the biggest problem with this is you can't refactor third-party code. This greatly restricts the modelling options that you have, because your TDD can only drive you so far, because TDD can only drive you to the boundary, so you can't refactor a third-party code. In this case, the solution is really simple. Square Mailer.deliver is a straight delegation through to whatever.view. But now our dependencies look like this. We have a single point of exit from our progress, which we can now refactor it, because now clearly what it does, you'd have somewhere to document it, and it actually turned out to be very useful. We wanted to add some new functionality through Mail delivery, and we now have the obvious spot for it. Notice we've added something new to make it less complex. That's really helped our productivity. Almost about the run out of time, but I've got five minutes or so. This is a different example where we had article and research, and we modeled it a certain way. This is not a square. This is a pretty different project. And this looks kind of weird, but it made sense at the time, because research and article both had very, very different relationships, and they stored different data. But we thought, ah, this is a bit weird but we'll run with it. What happened, though? And this wasn't a fault of this particular modeling. Just the way we write Rails apps, we tend to do this. The problem is the thoughtful models. We do things like we have a draft controller, and we create a draft, so we talk to the draft, active regular model. This is what screw is over with this model. When we needed to read back for it later and move the data around, having done this and trying to untangle these dependencies, this is what made this model very, very hard to deal with. I'm sorry I was kind of skipped on the detail there, but this is my way of introducing this concept of an aggregate. And so this is managing, I forgot to introduce this section, this is managing dependency inside your code base, so you're not dealing with external dependencies. I didn't know about this concept when we did this modeling. I wish I had. This is from a book called The Main River Design. Key point is we have a, we're treating a cluster of objects as a unit, and the unit has a root, which is the only way we are allowed to talk to that cluster of objects. In this case, meaning that we talked to the article rather than it's subcomponent graph, which gives us a dependency graph like this, which is much less tangled. I'll skip this concept, but do read into it more. It's very useful. I'm not very good at doing it yet, but one thing, because in this example it's kind of obvious, but in real apps I find very integral, but this is one tip that I found. Don't automatically add a belongs to whoever it has many associations you add. Try and see how far you can get with just one way dataflow through your actual models. You won't be able to hold up forever, but I found that a great way to get my head around. Anyway, let's talk about testing. That's an intro to this. Sorry, I've gone too long, so I've got three minutes. This is how Rails defines testing, which is both about SOLID and TDD and whatever. We don't do a very good job of TDD yet. We don't do the test driven. The driven bit, as a community, we're not great at. Test first? Yes, we're great. T, driven? No. And the one thing that I want to add to the conversation that's happening at the moment, Rails community starting to talk about, which is excellent, this verbiage, which Rails has, has to die. It sucks, and it provides very, very little value. And it's actively harmful. Two reasons. One, these don't actually, they just tell you where to put tests. These just tell you a convention, which is great for that convention of configuration, but a unit test is what you do for your models. Functional, it's what you do for your controllers. Integration, when you have more than one controller. But they're all kind of the same type of test, which doesn't put them in different folders. And the worst bit about this is where that we've co-opted terms, especially unit, which are very, very well-defined elsewhere in the testing literature. And because we've done this, we now have to talk about fast tests or database-less tests or things, a unit test doesn't touch the database. This is the definition used by everybody else except the Rails community. If we use this definition for unit test, think of what we could do. We can now, we can co-opt all these other techniques that have already been developed by other frameworks. And we wouldn't have to go through this translation phase of saying, oh well, it's a unit test, but it's not really a unit test, but it's kind of a unit test. The way we use language is very, very important. And the language that Rails provides us is actively harmful to think about. Here's another set of definitions. Except it's integration and unit test. This is what I have found the most useful as far as structuring my apps. If you look at the Rails definitions, all three of them collapse into the integration level, which means that using the Rails testing terminology, we're eliminating both acceptance and unit testing from our record point. We're getting better acceptance and unit we started to do, but imagine if you had spec acceptance, spec integration, spec unit, how would you now develop something? Language changes how we think and language shapes how we code. I think the one extra thing I want to add to this debate is we really need to rethink how we use these terms. We're trying to reinvent the wheel and it's not working. I've kind of lost stuff. There's nothing new in this talk. All the concepts I've prevented, I've just nicked from the course and code and screencast and whatever. This is, if you find this like later, there they are. Do read these if you haven't. That's pretty much it. I heard this quote about, I don't know, two years ago from this talk and it was a revelation to me. We kind of assume that code rots as it ages. I'm not just taking this for granted, right? Old code sucks. Deal with it. Sandy was talking about apps that she's working on and she has to get more fun to work on at the same time. This to me seems like a holy grail. I don't know how to do it yet, but I'm trying and hopefully you will as well. Thank you very much.