 Before we get started I thought it might actually be quite nice to get to know you a little bit And I realized this is just after lunch and everyone's feeling a little bit Sleepy and full of food. So if you have been professionally or otherwise programming in Rails in 2018 Can you raise your hand, please? Okay, good If you were pro keep your hands up. Come on. This is exercise If you were programming in Rails in 2017 keep your hand raised otherwise put your hand down 2016 2015 That's a lot of people been programming in Rails for a few years 2014 13 12 11 10 9 2008 2007 2006 2005 That's is there two of us 2000 2004 The very first release. Okay, that is amazing. You should come and give this talk down Round of applause for the old-timer. I feel so young I was gonna go down to 2003 and then say you're either a DHH or a liar, but I'm glad that Everyone was honest Okay, let's get started. We can take the lights down now. Thanks My name is James Adam and I have been using Rails for a long time I'd like to share with you some of my personal history with Rails Hopefully it won't be too self-indulgent and hopefully I won't bore you before we get to the important bit at the end I also want to apologize apparently every time I come to America now I get a cold So if I cough or sneeze or anything, please don't think I'm gross Anyway, let's get started. So the room is nice and dark I want you to sit back and relax get as comfortable as you can You can feel your limbs and eyeballs Getting heavy all your worries are drifting away, and I'm gonna gently regress you all the way back to 2005 This is me in 2005. I was just finishing a PhD I'd actually discovered Ruby a few years earlier and I'd totally fallen in love with it I rewrote almost all of the code that I was using to do my research in Ruby. I Was so excited about Ruby when I first discovered it that I sent this in an email to my friends who had all gotten proper jobs When they'd left university writing software most amusing Java at the time Now the nature of PhDs is that when you finish you've become a deep expert in a field that's so narrow It can be really hard to figure out how to connect it to anything else to anything in like the real world And I was worried in 2015 2005 sorry that I was gonna be in a similar state professionally because I'd fallen in love with Ruby But there's you know You can't get a job doing something that you love all the time And I was really worried that I was gonna have to go back to Java or C++ in order to make some money But I was actually incredibly lucky because just as I was finishing my PhD this excited Danish guy Started talking about a web framework. He was writing with the weird name Ruby on Rails But I should mention now. I'm gonna show a lot of blog posts. You don't have to read them all I'm gonna highlight all the interesting bits. So don't try and read everything I show So long story short a job in London got posted to the Ruby mailing list And I took the seven-hour train journey down from Scotland to London for an interview and then a few months later I was working in my first job Using Ruby being paid to use Ruby and a very early version of Rails I think it was 0.9 at the time. So that's maybe a couple of point releases after you Now our team was like a mini agency inside of a much bigger media company And the company used a tangled mess of Excel spreadsheets to run their business in like multiple departments using All kinds of crazy stuff inside those spreadsheets and it was our job to build nice clean web applications to replace Some of those spreadsheets at least We were a small team. I think never more than four or five developers But we'd work in a whole range of applications often at the same time Now these days Rails in particular with Rails 5.2, which was just released a few days ago It gives you everything that you might need to build a web application But it's easy to forget a time when Rails had almost none of the features that we take for granted now a time before active storage a time before encrypted secrets before action cable before turbo links or the asset pipeline before resources or rest or even before rack These are the headline features of Rails 0.13 in spring 2005 Migrations were brand new. Can you even imagine a version of Rails without migrations? And in lots of ways Rails 0.13 is similar to modern Rails. It has all the kind of MVC things that you'd expect models with associations and controllers with actions and view templates and action mailer But in other ways, it's actually more similar to Sinatra Than to the Rails that we have today And for example, if you install Rails 0.3 and all of the gems it needs then in total You have about 45,000 lines of code But if today you add Sinatra and active record and nothing else to a gem file and install that you end up with Over a hundred thousand lines of code. So Rails 0.13 is like less than half of Sinatra plus an ORM But the core philosophy of Rails then is exactly the same as it is now convention over configuration using the expressiveness of Ruby to try and make web development fun and Providing most things that most people needs to build a web application So it's summer 2005 Rails is at 0.13 and in our team we're building all these applications And we realize that we're building the same basic features again and again Specifically things like code to manage authentication. That's controlling who can access your application and Authorization so controlling which of those users have access to things like an admin dashboard and stuff like that And I think we had at least four applications under development at the same time Which is around about one per developer and so it just seemed counterproductive that each one of us would be building the same feature Each in a slightly different way, but for no good reason Possibly introducing our own little bugs that would be different in every application and generally making it harder for us to Collaborate on these applications or move around if someone was away After all a big part of the rails philosophy was and is dry. Don't repeat yourself And all these authentication and authorization features weren't particularly complicated or interesting They had the same stuff you might expect today logging in with an email log out something about forgetting a password a User model and a table in the database to support that and maybe some mailer templates to handle user lifecycle So it occurred to us that what we really needed to do was to write this once and Then have a way of sharing it between all the applications that we were writing So that we could all benefit from bug fixes and new features and be able to rotate around these applications in our team more easily So in modern rails, we typically share code between applications by using a gem with a rail tie inside a rail ties just a way of Configuring how you hook into rails initialization But these weren't added to rails until 2009. So they don't exist yet. We're in 2005 Before rail ties we had plugins, but they didn't appear until rails 0.14 We are 0.13. So these don't exist yet either. So we don't have access to that And even using regular Ruby gems, which did exist in 2005 was problematic because we didn't have bundler We didn't have ways of like locking down gems to individual applications So it was fairly typical that you would deploy multiple rails applications to the same machine But if you upgraded the gems on one, it would upgrade the system gems and that might end up breaking your other application rails did add some Rake tasks, I think for freezing gems, which is like a precursor to bundler, but they didn't appear until rails 0.14 as well So we couldn't use that so without any existing mechanism. We had to roll up our sleeves and invent something ourselves so in the late summer of 2005 we extracted all the login and authorization code including controllers and models and views and so on into a separate repository and then wrote a One file patch library which when rails loaded makes sure that our controllers and models were in the load paths for rails and Then made a few monkey patches to rails internals to make the rest of the stuff work like you would expect Originally, this patch was called frameworks, but pretty soon after it got renamed to engines The name engines was actually the idea of Sean O'Halpin It was my boss at the time and has been programming in Ruby probably longer than either of us So that was summer of 2005 in October of 2005 a few months later Jamis Buck added an experimental plugins feature to rails now plugin is just a folder of code that contained a lib directory and a file called init.rb and rails plugin mechanism at this time Just iterated through a series of folders in vendor plugins looking for init.rb files and then loading them and doing whatever was in them You could do anything And we spotted this feature being added to rails and so I emailed Jamis and DHH to say hey We've been working on something similar. We'd be very happy to contribute And we got a very nice reply the gist of which was that sounds interesting. Why don't you package that up as a plugin? And then we'll see how people use it So that is exactly what we did Resulting in the engines plugin Which let you treat other plugins as these kind of MVC slices Which could be shared between applications and this is the very first home page for the engines plugin that was hosted on Rubyforge I also released the first engine the login engine which was super simple It just wrapped up code from one of the existing authentication generators That we had used in our applications along with a few other tweaks that we found useful And this was November 1st 2005 And people got pretty excited like kind of super excited which is really exciting for me being the person who was like proposing this idea There was enthusiasm for the demonstration login engine that we'd released and it seemed that people understood the idea behind what we were doing Somebody even tried to turn typo Which was the first really popular rails blogging platform into an engine so that people could have blogs to their applications and People got specifically very enthusiastic about never having to write their own login code again And I think some people thought that they would never ever have to do that That these engines were going to just like solve this problem for them forever But then somebody got so excited that they started talking about engines within engines depending on other engines And I think it was this idea that ultimately pushed DHH over the edge and about ten days later He wrote the first blog posts about engines on the official rails blog Now in the posts David talked about his distrust of the dream of component-based development And that it's better to build your own business logic than try to adapt something that someone else wrote And that we shouldn't expect to be able to plug in or swap out these high-level components like forums or galleries That sort of stuff and never have to build them ourselves And I agreed with him But tried to clarify that what engines were good for was taking some code that you've written and then sharing a bunch of Applicate sharing it around a bunch of applications that you are writing As long as those features are relatively isolated from the rest of the application And I think David agreed with that And I can see his perspective when you're working on a single application like say base camp Then you probably can and should develop as much of the business logic as you can for yourself So that you can provide the best experience to your users But if you're working on three or five or ten different applications at the same time Then chances are that the balance of value versus the cost of sharing might tip the other way So I was pretty happy with that conversation It seemed like we generally understood and agreed with each other about the benefits and the dangers of what I was proposing I shared an idea and David had just expressed a bit of caution But a bunch of people had become super excited by it and maybe a little bit too excited And then a lot of people on the other side took David's post as a declaration that the idea was fundamentally bad and To be avoided at all costs And so that's basically how things played out for the next three years Every now and again somebody would write a blog post saying maybe rails need something like engines or engines are actually that bad Only to be met with a response like oh, didn't you know the engines are too much software whatever that means And like really bad and then I would write a comment to be like I think it's a bit more complicated than that and then maybe the author would add some clarification to their post But by that point it's too late and you've got people commenting that rails engines are actually evil This is real. This is all real. I call this time the wilderness years and During this time I tried to respond to criticisms of the engines concept with varying degrees of success It was occasionally quite frustrating. I Spoke at a bunch of conferences about plugins and sometimes engines And I also tried to gently steer the development of the plugins mechanism in rails To make it easier to do the things that engines needed it to do so that was adding things like controlling plug-in loading order giving plugins a more structured access to rails initialization and configuration and stuff like that plugins became very popular and Went from originally being shared as links on a wiki page to having their own directory where you could search for different plugins to help you With the application you were building and you could leave comments on plugins and so There's an example of a fun plugin that I wrote for one of those presentations You can see that even though maybe I'm frustrated inside. I am actually having fun And when if I did mention pinch engines gin those presentations I tried to always mention that there were valid use cases, but yes You could get yourself in a mess, but that doesn't mean that people should never ever use them And I'd hope that speaking at those conferences and writing those blog posts if not Convincing people totally to what I believed would at least soften them to the idea that engines weren't a hundred percent evil But then on the official plugin directory you'd get someone tagging the engines plug-in as shit And the cycle would start again. I still don't know who did that Maybe maybe we'll find out one day So some people would go to great lengths to explain why rails engines were bad As an aside, there's a thing called better just love headlines Which says any headline that ends in a question mark can be answered with the word no But I'd write like a short comment to try and respond to each of their points and hopefully clear up any Misconceptions about what the engines plug-in was and what engines were good for And in this particular case though, what was super confusing is the same people then released their own plugins to try and do basically the same thing as engines I've never really understood why that happened and the wilderness years lasted long enough that some companies actually wrote plugins like Engines themselves not realizing that engines had ever existed So I had any Conversation with Brian and we agreed that we should probably try and like coordinate our efforts instead of both having to fight this battle again So all this is happening between 2006 and 2008 during which a new Ruby web framework called Merb appeared And it was designed to be extremely fast Largely because it didn't do very much And we particularly good at handling many simultaneous requests and things like file uploads And unlike rails which at the time was a relatively tightly coupled set of web frameworks Merb was designed to be extremely modular so it could for example support multiple ORMs It was also designed to have clear and stable internal APIs Since much of the Merb framework was written as optional plugins that you could load or not load depending on what you needed And one of the developers whose most involved in Merb was you who the cats who eagle eyed people will have spotted was generally sympathetic to the concept of engines And so it's probably not surprising that in 2008 Merb introduced their own Implementation of the idea called Merb slices to a generally positive response from the Ruby and Merb communities But it's not a huge surprise to me that this is how the most popular rails podcast at the time chose to frame that Kind of like rails engines, but doesn't suck exactly And I don't blame the presenters for thinking or saying that it's just a representation of the opinion Of the rails community as a whole at that time This is a painting of Sisyphus Who in Greek mythology was cursed by the gods by being forced to roll an immense boulder up a hill Only for it to roll down when it nears the top repeating this action for eternity and these days It's a common image invoked to describe tasks that are both laborious and futile So we come to the end of 2008 Rails is about to reach version 2.3 and the controversy has largely died down people who found engines useful We're just getting on with it And I hope they were happy and the people who felt that engines were evil seem to have forgotten that we existed So you can imagine my surprise when I received this email from DHH with the subject line I repent I think I actually became giddy after I received this I may have laughed maniacally Rails core had decided that engines weren't evil and that they were going to be integrated into the framework My work was done Okay, it's not It's not really that simple, but without going into a huge amount of detail Rails 2.3 Is plug-in system absorbed some of the behavior the engines provided They could provide Controllers views and most other types of code and this was released as rails 2.3 in March 2009 In the same time Merb and rails decided to merge and rails 3 would be the result and the goal of doing this in part was to Establish some clear and stable API's within rails so that other libraries and plugins could rely on them and they wouldn't break Every time rails was upgraded and this is a fairly significant rewrite of a lot of the core parts of rails in order to create those API's Yehuda and Karl Lers did much of the work and as part of it They decided that rather than having a rails application and then these engine things inside of it that looked like rails applications and access to the same hooks and Configuration and so on but instead the outer application itself should just be a rails engine with a few other bells and whistles to make it work So I guess the engines inside of engines person actually got their wish after all and this was released as rails 3.0 in 2010 And finally with rails 3.1 in August 2011 the last two bits of work that the engines plug-in did managing Migrations and assets got merged into rails and the plug-in that I had written was officially deprecated My work really was done at that point So you've probably heard of the Gartner hype cycle Which is a way of understanding how technology trends evolve you have the initial creation or discovery of the technology And then the peak of inflated expectations where everyone is like super excited about it and imagining like having jetpacks Or living on the moon or something Then you enter the trough of disillusionment when it turns out it's actually much harder to build a jetpack than we thought And there's a bunch of things that aren't ready before we can start building it Then you start to climb the slope of enlightenment as we figure out all those problems and like understand how you can build a jetpack without setting your legs on fire And then finally you reach the plateau of productivity where everyone is zipping around on their jetpacks And we look back at old movies of people moving around using their legs and laugh at how quaint they seem I think we can use a similar cycle to understand how the rails community reacted to the engines concept So for engines it took just under six years from idea to acceptance We have the same starting point and the same peak of inflated expectations like I'll never need to write login code again But then if you just give me a second, I just need to adjust the axes here. We enter What I like to call the trough of received opinions Where some big names in the community have been like whoa, whoa, whoa, and personally we've never used the technology But we've heard that it's bad. So basically it's the worst thing ever And then for about three years we scrabble up this slope of fear uncertainty and doubt where quietly privately some of us are like Hang on I actually would quite like to share some of this functionality between the apps I'm writing but then you try and figure out how and you discover some old blog posts that say it's evil So you just give up and then finally we reach the plateau of oh are those still a thing And as you can see at the end of the cycle we end up like basically where we were But at the very least I can finally put the boulder down and stop pushing it up a hill So that took about 22 minutes if you'd like a nice summary I found this quote in the rails 3 in action book There was a lot of controversy surrounding engines and James spent a lot of his time defending the decision to develop them Since then however the community has grown to accept the idea Good so What changed in 2008 well, I actually only realized this as I was writing this talk But it's quite simple in retrospect So rails was originally extracted from base camp the software the DHH built and still works on today and at the start of rails life base camp was the Only rails application that David worked on but between 2005 and 2008 37 signals added another three flagship applications along with a couple of other small ones and their small team I think it was for developers at that point had to build and support these applications All at the same time And that sounds kind of familiar doesn't it? Really so that was just my theory amazing That's I might need to kind of go into therapy to fully process that But we'll keep going from now So in the rails doctrine which is a somewhat provocatively titled document that David wrote about two years ago There's a section called provide sharp knives and what that means is that with some of the tools that rails gives you It's definitely possible to get in a mess, but instead of protecting you from misusing them by withholding them all together rails Says that you should trust yourself to use them in an appropriate way and Concerns is one example of this sharp knife if you've ever used or read about concerns Some people think that it encourages sweeping complexity under the rug While others think that used appropriately that's not a big problem and the benefits are where the costs and That's exactly what the engines concept is. It's just a sharp knife For around six years It was a little bit too sharp to be kept in rail silverware drawer But now it seems like perhaps we can be trusted with it and these days a lot of popular libraries are engines You probably all used or heard of device, which is an authentication engine the spree e-commerce platform is Distributed as an engine you can get content management systems like refinery CMS Which is an engine and even the new active storage feature in rails 5.2 is implemented as an engine So that's the end of our trip back to 2005 We're now back in the present with active storage and this is a good moment to take a stretch if you need it But before we start the third act I want third act. I wanted to mention one thing that has nothing to do with rails or engines Most of what I've talked about happened at least 10 years ago and As you can tell I have some feelings about it So when I was writing I wanted to make sure that I hadn't inadvertently distorted any of what happened So I wanted to find the like actual articles and things and all of the comments and posts that you've seen are real They're just they're exactly as they were But when I try to actually find them find all the original news group posts and articles and blogs and so on What I found is that almost all of the sites that I've referenced are either totally gone or only partially available The comments on the rails blog are gone loud thinking is gone Ruby Forge is gone The rails engine site is definitely gone because I'm not paying for that anymore I Think that history is interesting and important and it's kind of mind-boggling to me the information that's only 10 years old might Without archive.org just be lost forever I think that's insane, especially in like the information age. How can that possibly be? So if you can please support archive.org they accept donations on their website, and I genuinely believe they're providing one of the most valuable services on the web today Okay Back to rails So at the start of this talk, I did say that I didn't want this to be too self-indulgent And I don't want to paint myself as some kind of misunderstood genius or hero who's finally proven right And I'm sure that there are many other stories like this one in many other open-source communities But what I think is interesting about this journey is that it shows that the history of rails can be viewed as a history of opinions Rails is opinionated software, which is great because it saves us a lot of time by allowing us to offload a lot of decisions about building software in exchange for some implicit constraints and Following those constraints is what we sometimes call the rails way And some of those opinions are about how we use Ruby as a programming language about how you should express behavior at the level of a single line of code An example of this Of the methods that rails adds to objects like string or array Objects in rails applications tend to have a lot of methods And some people believe that it's better to minimize the number of methods on an object But it's rails opinion that the trade-off is worth it in order to be more expressive And there's nothing wrong or evil about either of those positions They're just two different opinions Our opinions are a more architectural level and are ultimately about how we ought to structure the applications that we're building with rails An example of this is if you build your URLs and controllers in terms of rest and resources Then you'll be able to use a lot more of the abstractions and high-level mechanisms that rails provides But if you like to add a lot of custom actions to your controllers, then rails can't stop you and it won't stop you But you'll have to do a lot more work yourself to wire these things up Well, that's not the same as saying if you don't use resources, then your code is bad It's just a guiding opinion that rails has And what might not be obvious is That over the 14 years of rails life so far those opinions have not always existed and they have not always stayed the same The rest verbs and resource-based architecture weren't a part of rails for two years An inline JavaScript was fine until 2011 when rails decided to use Unobtrusive JavaScript instead and if you wanted to send an email for example when a user signed up up until rails 4.0 You probably would have written an observer and used that to decouple User creation and everything that that's about from email concerns But they're removed in rails 4 and so in the modern rails where you probably handle that by writing a concern Which mixes in a callback directly to your user model? And I think a particularly interesting example is active resource It used to be a part of the rails framework It was an evolution of the action web service framework which supported soap and dynamic wisdom generation and XML RPC and all the acronyms that David mentioned as WS death star and his keynote yesterday Active resource let you let's you load and save remote data using JSON over HTTP Using the same Ruby methods that you'd use if you were interacting with an active record instance And it made it easy to build things like micro services And so I think acted as a signal that you could and should do that But it was removed in rails 4 which might be the one of the first indications of the Current opinion that the majestic monolith is a more productive way to work overall And the purpose of highlighting these changes of opinion is not to say that DHH or anyone who was or is in rails core is Frequently wrong is to show that even in the world of opinionated software opinions can change Fundamentally what is and isn't in rails is driven by the needs of the people who write it And so to a greater or lesser extent that means people building applications like base camp But not everyone is building an application like that I think more and more of us are working on rails applications that have been around for a long time in some cases more than 10 years now and Those kinds of applications have different needs and experience different pressures Than one where the developer controls all the requirements and is free to rewrite it if they wish by any time that they wish According to built with comm there are almost 1.1 million sites using rails right now So it's statistically extremely unlikely that base camp as a piece of software or as a company Sits at the exact center of all those different needs and pressures and trade-offs that those other applications and companies might have Right now there are differing opinions in the community about what the future of rails might include the majestic monolith versus Microservices concerns versus smaller object composition system tests versus unit testing and stubbing and these Explorations and tensions are good because we need people to be pushing the boundaries of the rails way to figure out what's next if We all just sit back and wait for a relatively small group of people to just tell us what the future of rails is What it looks like then that future will only be suitable for their particular unavoidably limited contexts Now in 2014 37 signals changed their name to base camp and returned to maintaining a single application So some of the motivation within rails core for things like engines is naturally going to diminish and that's understandable It's an itch that they no longer have But I wonder how many other software itches there are which base camp doesn't experience But hundreds or thousands of other application developers do And I think we need more voices sharing their experiences good and bad with the current rails way And we need people to build things like trailblazer and rom and hanami and dry rb And we need then other people to try using them and learning from them And probably none of these projects are ever going to usurp rails But they might contain ideas about how to build software or how to structure web frameworks which are new and useful Unlike murb they might end up influencing rails towards something that could be better for all of us They might have already found some Conceptual compression to use a phrase from yesterday That we can either adopt or adapt for ourselves And there's no reason why the people doing that exploration can't be you Because no one else is going to do it You are the rails community. You are the people using rails all the time Who better than you to spot these situations where a new technique or approach might help? Who better than you to try and distill that experience into some beautiful expressive ruby code That captures a common need You can be the crazy ones the misfits the rebels the trouble makers the round pegs in square holes The ones who see things differently As it says at the bottom of the rails doctrine. We need disagreement. We need dialects We need diversity of thought and people it's in this melting pot of ideas will get the best commons for all to share Lots of people chipping in their two cents in code or considered argument Okay, wonderful rails now embraces all manner of opinions under its big tent But what happens when you have your idea? But people don't quite understand it immediately and things get a little bit out of control and suddenly people are decrying it as evil Well, I feel for you genuinely because when I was working on engines the main people the main way that people express this Kind of opinion was in the comment form of a blog But these days we live in the age of the tweet where many people don't think twice about unleashing a 280 character salvo of hot Takes out into the world, but I'm not sure that we live in the age of considered argument anymore So what can we do? Well two things Firstly as consumers of open-source technology I think we could all try our best to avoid sharing opinions like that if you've had a bad experience with a technology or a technique Then that is totally valid and you can and absolutely should share that experience But just don't do it in a tweet or if you must do it in a tweet at least try to be a little bit balanced Or even better than that start a blog or write on medium or anything and write as much as you can About your experience and your context and then share a link to that on Twitter. Where was I? Yeah, this talk sucks But if you really hated it don't tweet about it write a blog post and then post about that on Twitter Okay Secondly if you are lucky and generous enough to actually try to contribute a new idea to either this or any community Try not to become Demotivated if people don't necessarily understand the point at first. So this is that first blog post about rails That I showed at the start of the talk on the 37 signals blog in early 2004. I have not edited this Let's look at the very first comment Why use Ruby when you could probably have developed this thing in less time with PHP. Is it just some desire to be different than everyone else? I think what this shows is that the value of why you're doing something differently is often not immediately obvious to other people You will have to patiently explain it. You should see the number of comments that they've had to write after this Sometimes again and again and sometimes for years and years and you won't be able to convince everybody But you might reach somebody who finds your idea interesting or useful And they might reach somebody else and before you know it lots of people are getting value out of your idea And it might end up making a big difference after all So there's one last thing I'd like to say When you make something and it receives criticism on the internet, especially so basically criticism from strangers It can be very hard to deal with and sometimes so much so that we might stop creating altogether or never even try And when I was researching with a stalker found this old blog post by DHH from 2006 That I think captures a good way of thinking about these things. So I'm just going to paraphrase it for you View your idea or the thing that you've made as a pearl not a diamond When someone responds to your idea and points out all the flaws or the situations where the idea doesn't work for them That's okay because what they're asking for is a diamond. They want something that is flawless. They want something that is perfect But you need to remember that however that was expressed vitriolically or constructively or somewhere in between that it's not your job to make the diamond for them All you can do is give them the pearl that you are offering And if that's not good enough Then fuck them Thanks very much