 Minor tragedy, I lost my dongle for my clicker. So, a little bit late, but let's get started, welcome. Today we're going to dig into Active Record with a couple detours. In the lower right, there's a link to the slides if you wanna follow along. If you have press P, you can see some presenter notes and if I actually get some links and notes that I won't cover in the talk. My Twitter handle's in the lower left corner. Feel free to tweet at me or about me and use the hashtag RailsConf. So, we'll focus on some major issues with Active Record. Look at some alternatives, but then we'll talk about why you might not wanna use those alternatives and then we'll talk about a potential solution and a pattern that I think we should all be using. Who here uses Active Record? Okay, most of us. Who hasn't used Active Record? All right, I don't see any hints for that. Who's used a different Rubyorm? Couple hands. How about a non-Rubyorm in a different language? Couple more hands. Who loves Active Record? A few hands. Who hates Active Record? Anyone else have a love-hate relationship like I do? All right, actually more with that. So, Active Record is the 800-pound gorilla and odds are, if you're gonna work on Rails, if you get hired to work on Rails, you're gonna be using Active Record. First, I wanna make sure that everyone knows what an orm is. So, Ruby deals with objects, obviously, and SQL databases deal with relations. It's actually something called relational algebra that they work with. Sounds pretty cool. I'm not sure there's any mathematical foundations for no SQL databases. So an orm brings these two sides together. It maps between objects and relations. Note that there's an impedance mismatch between the two sides. What works well on one side might not work well on the other side. So some data structures can't be mapped one-to-one. One of the canonical examples of this is a tree structure. Really easy to do in OOP, but there's several different ways to do it in relational algebra and it's difficult to map between those two different ways. I gave a talk at RubyConf 2015 where I dove more in depth into what an orm is, sort of found the essence by building one in 400 lines. So Rails Active Record is based on the Active Record pattern. Here's Martin Fowler's definition. I'm not sure if he's the first one to come up with it, but he did document it in the patterns of enterprise application architecture. Note that he lists three separate things here. Wrapping a database table, encapsulating database access, and adding domain logic. Now you can argue that wrapping and encapsulating are pretty much the same thing, but domain logic is clearly a separate concern. Having that and in there is an indication that we might be violating the single responsibility principle, SRP. So here's a UML user diagram, or class diagram of the Active Record pattern. Note that there are two different kinds of things going on. Find and save, deal with persistent storage, they're above that line, and below that line we have name, agent, and address that deal with domain logic. The biggest problem with Active Record, I have that it encourages bad engineering habits, mostly because it violates the single responsibility principle. It commingles that persistence and that domain logic. Separation of concerns is important. Just like Rails separates MVC, Model View Controller, into separate concerns, we should probably be doing that with the model itself. And as your project gets bigger, Active Record's flaws become more apparent. I find it is about 12 to 20 model classes, it starts to hurt, whereas if you're below that, it probably doesn't really matter to you. Active Record is big, it's about 40% of Rails, and that size is another symptom of the single responsibility principle possibly being violated. Tries to do too much in one place and it conflates those multiple concerns. So, I'm showing some stats here from Rails 5.2.3, Model with One Field, and that came into over 100 instance methods and 600 class methods. For comparison, Object has 86 methods itself, but string and array have about 250 methods. So the number of instances is pretty bad, you're not gonna remember most of those, or a lot of those, and granted some of those are dynamic, you're gonna know that if it has a field, it's gonna have a few things related to that. But there's a lot of things going on. The number of class methods is really bad. So class methods have several issues, which I think I will cover later. All right, so the other thing I find super frustrating about Active Record is relationships or associations are defined in the model, like your has many and belongs to, but attributes are defined in the database schema. I think that's a terrible abuse of the dry principle. Dry says there should be only one place to look for any piece of info. I feel like attributes and relations are similar kind of thing that you should look in one place for. And the clue is I often wanna see them at the same time, but putting those related things in different places seems really counter to the dry guidance. So, you have to look in two places for all the details about a model. This is the case where there's too much magic for me. There's some workarounds. There's a model annotations gem. There is an add-on package to toggle showing the model's attributes from the schema. But unfortunately, that's currently broken for me. So attributes API actually came out in 4.2, but it wasn't publicized into Rails 5. But we have to use it and hardly anyone does. Does anyone here actually use annotations in their models in ActiveRecord? Decent number, all right, that's good. Thank you. So I also released a couple gems, actually, to let you define ActiveRecord models before the attributes API was a thing. One was called Vertis ActiveRecord. Anyone seen this talk by Enola Bob, Architecture of the Lost Years? A couple, anyone lucky enough to have been there? All right, one other person. I was there, I'm the person that asked him a question, can you show us some code? And he said, you have to figure it out yourself. So you can't hear me on the video. So it's a seminal talk. Oddly enough, it was only given at once at a regional conference. I'm not sure why, maybe because he didn't give those details and I called him out. I don't know, probably not. He doesn't usually seem affected by that. Since this talk, and probably even earlier, I've struggled to find a way to get Rails to implement all those architectural suggestions. My last project, we used the Interactors gem to do the Interactors part. It's actually on the chart there, up there, his diagram. And that splits the Rails controller from the business logic. According to this talk, the fact that our app is a web app is sort of incidental. And so we should add the business logic separate from that incidental delivery mechanism. And Interactors can give us that. There's a pretty good Interactors gem and it works pretty well for that part. But I've never found a great answer for splitting entities in the database. Which if you look there, he's got entities on the right side and then an entity database, or entity gateway in the database. And this is the quest that I'll be talking about today. So after almost 10 years from that talk, Uncle Bob wrote a book on the topic called Clean Architecture. It's a pretty good book, but it really doesn't help me with this problem. It doesn't get into the details. Uncle Bob also has a blog article called Clean Architecture that's got a pretty good succinct explanation. So the first stop on my quest is the SQL ORM. I will not pronounce SQL as SQL because then it would get really confused. So this is the biggest surprise when I did research for an earlier related talk. And it's written by Jeremy Evans. Does he happen to be here today? All right, no, I don't see him. So he was the winner of a Ruby Hero Award a few years back. SQL has tons of plug-ins and they leveraged lots of database features, especially for Postgres. It supports almost any SQL database you can think of and I really like the documentation. SQL has two different APIs that you can use. It's got the data set and the model APIs. Here's the code used to set up the SQL for the next couple of slides. Not a lot going on here. Pretty much just creating a table. SQL syntax is really nice. Here's the data set. Look at line four. Note that the block lets you use bare column names and a greater than. That's pretty cool. Active record does not let you do that although there is a squeal gem that adds that sort of feature. The problem is it doesn't stay synced up with Active Record as well as it would like. And I think I've run in a few other bugs and problems. Datasets are innumerable with each element as a hash-like object. You can see that being used in line five. I haven't come across anything that SQL can't do well, which is pretty cool actually. It just doesn't fit the pattern that I'm looking for. So here's a higher level API, the SQL model. That's using objects instead of just a hash-like object. And you'd probably be more likely to use this layering rails. We'd like to use object-oriented program. Like Active Record, attributes are derived from the database schema. But also like Active Record, relationships has to be specified manually. Again, that thing that really frustrates me. I really like SQL. I wish Active Record was more like SQL actually. But SQL doesn't solve the problem I'm trying to address. So next dorm in my journey is ROM, the Ruby object mapper. This was originally meant to implement the data mapper pattern. Does anyone remember the data mapper library? Anyone use it? What a decent amount of people. So originally this was data mapper two. And 2013 they renamed it to ROM. And 2014 they moved away from object relation mapping altogether. So it's not really technically an orm. It just maps the data not the objects. Most of the work was done by Peter Solnicka. Not gonna try to pronounce it in his language. P-I-O-T-R, Peter I believe in English. It's similar in spirit and partly inspired by Elixir's Ecto. Does anyone use Ecto? All right, good number. All right. So you guys might find this a little more palatable than I do. So, let's see. Peter Solnicka also formally wrote Vertus, a really nice attribute declaration library. ROM is a bit complex to use. It has commands, relations, mappers, and you have to buy into this completely different paradigm and mindset. ROM's developers are responsible for the dry RB libraries, which we'll actually talk about a little more. It's really good at small, independent, low-level, composable libraries. Some of the leaders of this movement towards functional programming and immutability in Ruby are part of this dry RB and the ROM team. I find them a bit focused on the load-level details too much and I think that's why it takes them a long time to get their product out. But once it's out, it's really high-quality code. So, ROM relation, this looks pretty straightforward and we've got a model class called user that's empty and then we've got a user's class, which is a ROM relation and we define the attributes there in a schema block and then we have the associations in a sub-block of that. I kinda like that, that's nice. And then I've defined basically what's a scope in there. We could also tell ROM to pull the schema from the database, we would replace the schema block with a schema infer true, but then we wouldn't see all the attributes and it seems like this is the preferred way showing all the attributes in this declaration. To save an object, we start with a relation, we knew up a relation object. We pass that to a change set. You know, Ecto, this feels really familiar and that includes a create or an update. I don't know what happens if you get it wrong and it passes all the attributes as a hash. So like I said, we're not really dealing with objects, you'd have to do your object to hash if you're dealing with a Ruby object. And then we have to explicitly commit the changes. I think that might be nice, I'm not sure. So I found ROM to be really complex. Here's an overview diagram of their architecture. Honestly, I can't follow everything that's going on there. So I found that confusing. I really want to like ROM, but I found it too complex and confusing. I couldn't actually get things set up right to run the code that I showed you in those previous examples. So the last of my quest is the model layer of Hanami. Hanami is a full web framework and alternative to Rails. I have like everything I've seen. If I had a choice, I'd probably choose Hanami instead of Rails at this point just to try some side projects. So Hanami supports SQL, actually via SQL, memory and file adapters. It follows the data-driven design architecture. I'll talk about that a little bit more. So it has entities which are models without persistence or validations. It's got a repository, which is mostly like the class methods in active record model classes. So things like create, update, persist, delete or all find first and last. It's got a mapper, which is a declaration of how to map between database, records and object attributes. Here's the first part of a Hanami model. So we inherit from Hanami entity, which surprisingly adds only four methods, at least the last time I looked. It adds ID, ID equals initialize and then a class method called attributes that we're using there. And that's all it adds. That's pretty cool. The fault initializer takes a hash of attributes to set all the entities attributes. And types come from the dry types library. So that types colon, colon int, types colon, colon string, those are dry types. We could again let the model pull the schema from the database like active record. But I don't think that's that common. So persistence is done by the repository class. Note that things like where and order here are private. We can only use them within that query there on line three. Queries are analogous to scopes and active record. And here we're using it. We knew up an article from the article repository. We grade it and then we can go find it and find it by author. So I think Hanami is my favorite Rubyorm. If I had a choice, I'd probably use it in Rails. But it's not a very realistic option as far as I can tell. It requires everyone on your team to learn something new. If it's just you, that's not a big deal. If we have a team of, I think if we have eight developers on my team, probably not gonna work. Also, probably don't want to use it on a project that already has hundreds of models that's been around for I think eight or 10 years. So there's not much documentation on using it with Rails, nor with the other orms I talked about. And Rails, I don't assume you're using active record most of the time. And they may or may not work with another orm. So Hanami model implements the repository pattern. And the repository pattern represents a collection of domain objects. And in a lot of ways we can treat the databases in memory collection. We can kind of extract that even more than we do with active record. We do have something similar in active record and that's the class methods. The create, the where, the find all. And when you create a scope, that's basically also sort of the repository pattern. But again, it's stuck in that, in class methods. And those have some serious limitations. It leaves a procedural code instead of object oriented code. It often indicates that you've missed an abstraction. It limits your polymorphism. And it's hard to test and hard to refactor. There's a good article on code climate that talks about all the problems with class methods. And if you pull up the talk here and the presenter notes, there's a link to it. So here's the UML class diagram for the repository pattern. Note the arrows. The domain model is not dependent on anything. There's a clear separation of concerns. The domain model class handles the business logic. The repository class handles the persistence. So we could actually end up with more than one repository for a given model. Maybe you want to do sharding. Maybe you want soft-deleted things in a separate database. Maybe you want rewrite segmentation. Maybe you want in-memory persistence for tests that uses a different database backing or maybe in-memory backing. You might see this repository pattern with a third class, which would be the mapper class that handles the coercion between the database fields and the object attributes. So I spent several years looking for a way to have my cake needed to. I want to keep using ActiveRecord, but I want to separate my domain model from the database persistence. So one Saturday morning I was in bed, staying in bed a little late, and I was thinking about it again. Don't ask. I don't know why I need to think about those things in bed. But I came up with a solution that I thought could work. In Rails 3, they split out the ActiveRecord. They split out ActiveRecord into several modules. And I thought, oh, I can use those various modules that ActiveRecord uses and split them into the two sides. The funny thing is I think I misremembered that. I think it was ActionController that got modularized for breaking into pieces and picking and choosing which pieces. ActiveModel did get pulled out of ActiveRecord, I think at that time. But I don't think they were meant to be used separately. So it wasn't quite as easy to make this work as I expected. All the modules have a lot of interdependencies. And there's no real documentation on how to use each module and what their dependencies look like. So, but it turned out that the domain model is just most of ActiveModel. So I ended up calling that ActiveModelEntity when I originally called it ActiveRecordEntity. And then the repository side is most of ActiveRecord. So I'm gonna show the difference between using standard ActiveRecord and using ActiveRecordRepository, which is the gem I'm working on. So here's the typical ActiveRecordModel. This should be pretty familiar to you. We have a couple, we have belongs to, has many. We have a validation and we have a scope. And then there's some fields that we don't know about by looking at the code, unfortunately. Here's the same thing using ActiveRecordRepository. Instead of subclassing, I'm including a module. This is an interesting little thing that I, a pattern that I found. The module is actually dynamically generated through the call to ActiveRecord, or ActiveModelEntity method. So we can pass parameters. I'll talk about that some more when I talk about the implementation. Let's see, the module we're mixing in is ActiveModelEntity. The term entity comes from Eric Evans' domain-driven design. And an entity is an object that has an identity. So we could have two items with the same attributes but different IDs. And those would be considered different entities. But if we have two items of the same type that have the same ID, those would be considered the same thing. And there's actually something called an identity map in ActiveRecord. The other major difference is we declare attributes here, the name and types, which fixes my second biggest gripe. We still have the belongs to, we still have the has many. What we don't have is the scope. And if we had any instance methods, we'd have those here as well. So here's the repository for that same class. Again, we're including module instead of subclassing. Again, we could pass some parameters. We could pass the model class we're working with. By default, I am taking the term user repository, knocking the depository off and assuming it's user. We could specify the database table name. If it can't be derived, specify a primary key. And we could specify a mapping of database column names to entity attribute names. And the scope is on this repository side because it deals with the entire collection, not any individual object. So here's a typical controller with Rails in ActiveRecord. Note we tell the user model to save itself on line four, and that will return false if it failed to save. Here's the same thing with my ActiveRecord repository gem. Only two lines have changed. Line four, we explicitly test to see if the model is valid and then deal with that. And then line five, we tell the repository to save the model object instead of telling the model object to save itself. There's one caveat. If you've got a uniqueness validation, that can't be determined until you hit the database. So if you do have that, you're actually gonna have to catch an exception on the save. So here's a bit of the implementation of the entity model. So I talked about that pattern, the parameterized model pattern. Unfortunately, I think this is the simplest implementation possible. But basically we create a list of modules that may vary depending on what we passed. And then we create a module composed of those modules. So the self-composed module, not important to understand. More important is to understand that we're taking several modules and we're composing them together. And this, like I said, allows us to pass parameters. So as I said, I previously called that ActiveRecord entity but we're not using anything from ActiveRecord. So I changed the name. You see there we're just including an extending active model modules. That's hard to say. So here's the repository side. Again, I'm using that same parameterized model pattern but I haven't implemented anything on this side yet. This side is all ActiveRecord plus some custom code. We're mostly making the custom code mostly make sure that ActiveRecord still works despite the pieces I've taken away. But it still does use most of ActiveRecord but not quite all. So we've got some helper methods to calling it ActiveRecord. This method lets you do user colon colon repository.save and then pass a user object. This one's a bit tricky. We have to create an ActiveRecord model object temporarily to save and then we update the entity's ID when we save to let us know that the entity has been persisted. This is an implementation of ActiveRecord or ActiveModel persisted question mark which I think is required to be an ActiveModel citizen and for Rails to be able to deal with you. There are quite a few challenges, quite a few more than I expected probably due to the fact that I misremembered which things got modularized. It didn't occur to me for a while how to separate the modules. It turned out the unique side is all ActiveModel and the repository side is all ActiveRecord. We're not subclassing ActiveRecord base and that turned out to be really tricky. I spent hours trying to fix this. ActiveRecord uses that to figure some things out and includes info about the connection to the database and I also had to tell ActiveRecord that the repository class is not an abstract class. Currently I'm fighting with ActiveRecord relations. I'm getting an error that doesn't seem to be related to the code I added. That makes it really hard to troubleshoot. So I still have a lot of work to do to make this usable. Please do not use this in production. I'm not gonna use it in production. I'm not sure I'll even get to that point but it was fun and interesting to learn that maybe I can make it. The main part is testing all the way the relations work like cascading deletions, loading where I have to load all the relations and map them to objects and saving in basically the opposite way. We could do automatically create migrations because we have all the data we need in the model class, all declared there. I think the only thing I'm missing right now is indexing. Datamapper actually had that option. If you're into migrations, go see Matt Dzynski's talk on migrations right after this, a teammate and colleague of mine. He covers a lot of gotchas with migrations. I plan to look at those gotchas if I do get to the automating migrations. It's in the next time slot over in room F. So I need some help from all of you. If you're interested, please go star the repo on GitHub so I know if people are interested. The more people are interested, the more likely I am to complete the project. I'm easy to find on the internet or in person. Got the WeedMaps t-shirt on today. I made the repo in the talk easy to find and I'll have links on everything on the last slide and the links kind of link to each other. I'd like to thank you all for coming to watching. I especially like to thank my coworkers who watched and previewed this talk and provided some valuable feedback. If you like listening to me, I do a podcast on Agile called This Agile Life. We do it semi sporadically. I'm not always on, but we have resurrected it and we are recording podcasts again. A big thank you to my employer WeedMaps for sponsoring this talk. There's about 20 of us here. Most of us have t-shirts on and we are hiring big time. Come see us at our booth. We'll have t-shirts. Not sure exactly which ones yet. So the source of the presentation is on GitHub in my presentations repo. Easy to find. There's the link to the Active Record Repository Gym. You can also find that on my GitHub page. I put it near the top. And feel free to stop me in the hall if you have any questions.