 No, there's lots of good choices, it's hard. So many tracks and workshops going at the same time, and so I appreciate y'all being here. It's good to see so many seats taken. I'm Barrett Clark, I've done Ruby for about a decade. I like to run, I currently work at Saber Labs, where I do research and build prototypes. Really to try to make travel suck less, Saber is a travel company. And I like to be outside as much as possible. But we're not here to talk about me. Today, we're here to talk about Rake. So let's get started. Rake is a Ruby scripting utility, really used to streamline tasks, repetitive, tedious tasks. It can be considered a Ruby version of Make, the Unix build automation tool. Similar to Make, we have a rake file, like the Make file. But the rake file is written in Ruby, so you can do all these things in Ruby. It was created by Jim Weirich, who unfortunately passed away in February of 2014. And I still miss him. I miss his spirit, I miss his influence, his energy. This was one of his last tweets. There was a new TV show called rake. But it was confusing because it wasn't about Ruby or build automation, which is unfortunate because maybe it could have been a good show. Okay, rake can help us manage our databases. If you've done any rails, then you've probably used database migrations and you've used rake. And I would say, really, this is probably the biggest killer feature in rails, database migrations. So here we have a migration from the Rails 5 beta. This was C3. So, of course, we're going to make a blog. Oops, it works. We've got our timestamps created out of data. We'll have two fields, a title and a body. It wouldn't be great if all blogs were this short. These are just strings, so they're varchar255s. So we want to create our table. To do that, we'll use bundle exec rake dbmigrate. You'll note the inclusion there of bundle exec rake is a Ruby scripting utility, so you could just run rake. But we're in a Rails app, and we want this to work in the context of our Rails apps gem file. And bundler can help us with that. And so bundle exec rake dbmigrate. So we run it, and you see I'm going to run the create post migration, and it's going to create a table, posts, and we're good. Oh, but I changed my mind. Maybe you're indecisive like me, and you change fields, change the names, add fields. You don't really know where this is going quite yet. So I want to add a permalink to my table, because we want to tweet about all the pithy things we have to say, and we want people to come find our blog. So we could create a new migration, which is reasonable, but we can also edit our existing migration. And I'll talk here in a minute about why and when we make those choices. But let's just say that it's totally okay, and we're going to just put this new field in there. So there's a couple of ways that we can revert this thing and rerun it. The first, we can just use dbrollback, bundleexec, rake, dbrollback. And so that's just going to reverse the migration. So ActiveRecord understands how to undo lots of things that it can do. CreateTable is one of the things that it knows how to undo. If it's not something that is a reversible migration, you have to separate up and down separately, and so then it would run the down method. So we do that, bundleexec, rake, dbrollback. And we say, okay, we've got this createPostMigration that we're going to revert, so we're going to drop the table. And dropping a table means you're going to delete the table. And when you delete a table, you delete all of this data also. But that's okay, this is development, right? You wouldn't do this in production. The other thing we can do is bundleexec, rake, dbmigrate, redo. And this is great. It'll roll it back and then roll it forward again. So you can see we're going to take that createPostMigration, we'll drop the table, and then we'll create the table. So we're going to delete the table. It's going to delete all the data, and it's going to recreate the table. So when it does it, you'll have a fresh table that has all the stuff in it except for the data. This is super handy. I use this a lot. We've got this blog post, we've got the permalink, and it's going to go viral because we have incredible stuff to say, right? So we're going to add an index to the post's table on the permalink so that we don't table scan every time we look up that field, because table scanning is bad, right? But we've introduced a new database object. And so when we try to roll it back, ActiveRecord is going to say, okay, well, I need to undo all the things that I did. So I'm going to drop the index, and then I'm going to drop the table. Now, of course, if you drop the table, it would drop all of its objects if you just did it in the database. But ActiveRecord is going to be thorough, so it's going to explicitly drop everything that is created, and we're going to have a bad time. Don't worry, that guy survives. He's fine. So we need a different strategy. We're going to drop the database and recreate it, then re-migrate. BundleExecRake, dbDrop, dbCreate, dbMigrate. And you can see we can run multiple tasks serially one after another. And yes, this is heavy-handed. This is totally score-stirth here. But again, we're in development. We're doing early development. We just created this table. We don't really know where we're going yet, and so it's okay. So you're asking, Barrett, when can I change? Can I just change the migration and re-run it? And of course, the answer in computer science is always, it depends. So, limitations. New fields are totally fine. You don't have any new database objects, and so you can re-run that. New objects, so a new index of foreign key, or for whatever reason, you wanted to create a second table in that migration. That's when you'd have to drop the database and start over again. And if you've already committed and pushed, don't change it. That's a bad thing. So, I'll tweak a migration and re-run it when I'm in initial development, and I'm still trying to figure out what I'm doing. But once I've set that stake in the ground, then it's fixed, and we're going to have to create new migrations. But I don't want to create 10 migrations while I'm trying to figure out, you know, what needs to be in this table. So, let's keep going with rake and database management. And we're going to talk about advanced database seeding. When you create a new Rails app, you get a seeds file, and it's empty, and it looks like this, has these comments, and it tells you, I've got these tasks that I can run for you, db seed, db set up. And when you run, let's say you run the db seed task, well, it's going to take whatever's in there and run it. So, the example here is we're going to create a couple of movies, and then we're going to create a movie character. And that's great. But the problem is each time we run this, it's going to re-run everything that's in there. And Star Wars and all of the rings, you know, they're trilogies, but we don't need multiple Star Wars movies in the database. So, ActiveRecord has our back, though. Instead, we can use, instead of create, we can use first or create. So, where we have this where condition, when that is not satisfied, when it cannot find a record, it'll create it. Otherwise, it'll just return whatever instance or instances it finds. We're using first or create, so it'll actually return the object or nil. Postgres96 recently shipped, and we finally got Upsert, which is update or insert. So, who knows? Maybe let's keep our eyes peeled for updates in ActiveRecord to support that. But first or create is going to be what you want here, and then you can safely re-run this DB seed task. So, why not rake all the things all the time? Because we can run multiple tasks serially, one after another. Why not make sure the database is completely up to date every time? We can safely run the DB migrate task several times. And if there is nothing to be done, then it just won't do anything. And we can now safely run our seeds task multiple times, because if it doesn't have anything it needs to create, it just won't do anything. We can also create a custom rake task. This one's going to take a CSV file and load it in. The CSV file has a header row that happens to match the fields in the table, which is handy. I've got a custom converter there, so any field that is blank, the quote-quote empty string will be converted to a null that the database understands. So there's a bunch of stuff there. Let's go through it piece by piece. We've got a rake task, and it's in the DB seed namespace. You can nest the namespaces, and so that's how you get DB seed, and then the task is import airports. And it's going to load the rails environment. We're going to look and see if there are any airports already loaded. If there are any records, we're going to assume that this task is already run, and we don't have to do anything. We could do that first to create, and that would work just fine, but there are lots of airports, right? That would be a lot of lookups. It's going to create a lot of objects it doesn't need to create, and we're going to generate a lot of log, and we don't really want that either. So we'll just be naive and assume if there are any airports, then we're good. CSV out of the Ruby Standard Library is really handy. So for each, given a file name, we'll open the file and read it in line by line, record by record. We've told it that there's a header row, so it'll take that row, and it takes each row and basically makes it a hash. So the header, the field names will be the key values, and then the hash values will be the data from that particular record. And I'm going to convert it to a symbol just so that it looks like what I'm expecting a hash key to look like. I prefer my hash keys to be symbols. The custom converter there, this is also a really handy thing that CSV offers us. So if there is data in that field, if it's not null, but it is empty, empty string, quote, quote, then let's make it null, otherwise we'll just return the field value. So that way we get good null values that the database understands instead of a table chalk full of just empty strings. Nobody needs that. So we'll pump that hash into airport create because the field names are the same as the column headers. You can pump a hash directly into active record create, and it'll create the records for you. You're good to go. Again, we could have done first to create here, but I don't want to do 8,000 lookups. Wait for that to happen record by record and generate all of that log. That's just a lot of unnecessary work. So here it is again. Custom rate has to load CSV file and import it if it needs to be. To invoke that rate task, bundle exec, because we want this to run with Rails apps, gem file as context, rake, db seed, and port Air Force. It's in the db seed namespace, so that becomes part of the name. So we've been plugging away, and maybe we did do first to create, and so our logs are starting to get out of hand a little bit. So we have too much log. Well, we've got a rate task for that in Rails. So you take a look at your logs directory, and this one's actually not very big, but you can see that test log is significantly bigger than the development log. You're running your tests a lot. Maybe you've got a guard running. It's every time you save, it reruns your tests, and so it can quickly grow. So bundle exec rake log clear will go through and it'll open up and truncate each file in that log directory. And it won't give you any output. It would be nice if it told you what files it's clearing and how much space it saved. It won't do any of that. But Rails is an open source, so we could hack on it. So I don't know, come talk to me afterwards if you want. So let's rake all the things. Because we can run multiple tasks serially, why not make sure our database is up to date and our logs are clear every time? So bundle exec rake db migrate db seed log clear. Again, we can safely rerun the migrate task and we can safely rerun the seed task. Rails can also tell us stuff. We can see all the notes that we've sprinkled for ourselves throughout our code. Bundle exec rake notes. It'll go through and look at all the to-dos and fix-mes that you've put in all the files. There's also an optimize tag. I haven't actually used that, maybe I should. So we do that in a test project here that I made, bundle exec rake notes. And we see in that posts model I've got a to-do. You can see the line number is line two and then on line three I've got a fix-me that just says this is an example, fix-me. And then in the seeds file I've got a to-do for myself to create some blog posts. So you can see the line number and the file of each of your notable notes. But documentation goes beyond comments. Your app can tell you things about itself. What does it know how to do? How do you use it? That routes file and the routes, that's one of the first places that I look when I pick up a legacy Rails app. Or just a Rails app that I didn't write or that I forgot that I wrote. So let's look at a routes file. This is an old one from an old work app. You can see there we've got a couple of get actions on a gate controller. We've got a readings resource. A resource is probably an API that we've exposed for another app to talk to. And that resource gives us the seven golden actions for free, index, create, new, edit, show, update, and destroy. And then we also have to find a root route. So if somebody just goes, so you know, localhost 3000 or myawesomeapp.com, then they'll get something. So bundle exec write routes. We see all the routes and we see those to get actions on the gates controller. We see the seven golden actions for readings and we see our root route. Now that readings index is actually not going to return anything. It's just a 204, an empty okay. There's no body. So seeing the routes, the named routes, seeing the URLs that you have in your app, is really handy. We didn't specify any custom names in any of our routes, but you could do that. So then in your app you would refer to these, you could refer to these apps, these routes, as gate manifest, URL or gate manifest. I think route, I think you can do that. And so you could put that in link to or redirect all those things. As your app gets big and you have more routes, you might want to filter. Well because this is just a, you know, command line scripting utility, chain commands together. We can pipe the output into another command. And so we'll do bundle exec rake routes. We'll pipe that through grep with the keyword gate so we can see any route that has that gate keyword anywhere in the line. So we have two gate routes. With a big routes file, this is really handy. Okay we can also use rake to update our projects. And this is a little bit like sorcery. Bundle exec rake rails update. You run that and it's potentially heavy handed again. So it's going to go through and it's going to look at all the files in your app or maybe your app version and it'll tell you all the new files that it needs to create or any changes in files that you have already. So this is the Rails 4 syntax. Rails 5 they've changed it to bundle exec rake app update. So we run that on one of our apps. And you can see it's going to go through a bunch of files and say, well we already got that. It wants to create a new secret key. We don't care. It gets the environment development. And we see that the active job queue name, they want it to be changed. Instead of being camel case, we're going to snake case it. And do we want to accept it? Well I haven't made any other changes in this file. And so it's safe to accept it. You can see I say yes. Here are the options that you get. File by file yes you can accept. No you can choose to not accept it. Or you can just step on everything. Or you can see a diff. So what I like to do is get that diff. And if it's a file that I've made changes in then I'll decide how I want to approach. Maybe I'll copy in the change or maybe I'll just step on the file and look at a get diff and see what I had just stepped on. It's safe to run this and see what needs to be changed without stepping on anything. But you've also got your project under version control so if you do step on something, you can always revert. So remember rake is a Ruby scripting utility. So it's not just for Rails. We can do this off the Rails. All you have to do is include the rake gem and have a rake file. Similarly we can use bundler in just a plain old Ruby project. So here's a very simple rake task. Got a description. This task says hello. The task is hello and it's going to put hello. So we run that rake hello and it says hello. It's a very friendly task, right? Well at a second task world and I put them in a namespace now because they're similar tasks. A namespace is a way that we can group like tasks together and it becomes part of the name like you saw before. So rake RailsConf hello says hello. Rake RailsConf world says world. Got a third task here and this one has some dependencies. It doesn't have a block. It doesn't have its own body because it doesn't need to do anything in addition. So when you run this phrase task, it's first going to go run the hello task and then it's going to run the world task and then if it had a body, it would run whatever was in that block. So we run that rake RailsConf phrase. It's going to run the hello task and say hello. It's going to run the world task and it's going to put world. Cool. So now we have a fourth task. We're getting tricky here. This task takes in a parameter. It takes an argument. It only has one argument to rename and in that block, it's going to come in an args hash and it's going to come in a controller where you have the params hash and the args hash and the rake thing. You can call it whatever you want. You define the variables there for the block. The idiomatic thing is we call it args. So we're going to say rake RailsConf custom hello bearer and it's going to say hello bearer. Here's how we would run it in a Ruby script. You can invoke rake tasks from inside Ruby. It is require rake. Load the rake file and then rake task, the name of the task, dot invoke and then if the rake task has parameters then they would go in the parentheses there for invoke. This also shows how to use bundler in a plain old Ruby script. Again, just require bundler, bundler dot require. We could have had the rake file in the gem file, the rake gem defined in the gem file and then it would have been required but I wanted to show it separately. So we run that Ruby script in the context of our gem file bundle exec Ruby rake include dot rb and it says hello bearer because that's what we told it to say. And finally we add a default task there at the we bottom. If you just run rake it's going to complain because it says I don't know what to do, I don't have a default task. So we give it a default task. We run that rake file and so the default task is rouseconf hello. So we run rake it says hello. We can ask our rake file what's so great about you and it'll say with rake minus capital T it'll list out the tasks and it says well here's all the tasks that I have. This is where that description line comes in handy. Any description that you, any rake task that doesn't have a description it won't list. And there it'll still run you can totally use it. It just won't be listed out in the rake minus capital T. Rail ships with a bunch of rake tasks and it's just open source right? So we can look at them. They're in the rail ties gym and lib rails tasks and so those are all of the rake files that ship with rails. Because it's open source we can go play in the code. So here is that log task and you can see all it does it's really simple. It just looks in the logs directory gets a list of the files and truncates them. So this is where we would go if we wanted to do something like print out the name of the file as it was doing something do a file stat to see how much space we were saving this is where that would go. And we could submit a pull request if we decided that we liked it and maybe it gets pulled into so remember rake is an automation tool we can streamline common tasks so remember we did that when we raked all the things db migrate db seed log clear. Well we can streamline that. We can create a custom rake task that just does that for us. And rails has our back there's a rails generator for custom rake tasks so when you run one of the generators or when you just run rails g rails generate it'll tell you well here's what I know how to do and then you run the rails g for tasks and it says well here's how you use this thing. So let's generate a task put it in the db namespace and call the task streamlined and you see it'll create a file and live tasks and it'll be called db. rake when you do that you get this empty file there's a lot of boilerplate already taken care of for you you got a description you just have to fill in the to do and then you get the task streamlined it's going to load the rails and it's going to have an environment for you and then all you have to do is fill in the block what do you want your tasks to do we have everything we need it's perfect so I'm going to fill it in I'm going to say the description is well it's going to run all these things and then I change it to the hash rocket the older hash syntax because I just like that better but if you like the new syntax that's cool we can still be friends it's fine so this streamline will now have three dependencies we no longer need to load the rails environment because db migrate will do that for us so streamline will run db migrate db seed and log clear and we don't need it to do anything else and so we don't need that block we don't need that body we can ask our rails app now what do you know how to do and we can filter it for any rake tasks that has the word db in anywhere in the line and they can say oh hey cool I've got this new task rake db streamline it won't actually be highlighted I did that so you can see it's actually there and so we're good to go we can run our new rake task and it works so it's going to migrate the table so we'll create the table run the seeds we don't have any output for that and then look at the log directory it's empty so it did all the things we're good to go other streamlining TDM sorts of things see how the microservice set up one time and we used auth tokens to communicate between the services and it was a calculated token that had a time stamp in it to test one of the child services that had to have a valid token and it was kind of a pain so I made a rake task to calculate a valid auth token so I could just use curl to test those Docker management before we had Docker machine back in like the fig days I wrote a big rake file that helped with all the container management and I actually still use some of those tasks to help with cleanup we saw loading a seed file into the database but we can also load a full production database into our local development database here's a custom rake task that will download production database from Heroku and load it into your local development database it's just a series of three tasks one is dependent on the other so let's break them down here's the first task and we've got capture backups it's going to load the Rails environment and it's going to ask Heroku to capture a PG backup Heroku is going to just go make a PG dump and store that somewhere we can then download we've got this download backup task that's going to go ask the capture backup task to capture backup and then we'll use curl to download that dump so we'll have it now on the local file system and it's going to ask load and it's going to use PG load, PG restore to load that dump file taking in all the variables from the Rails environment and then it's just a temporary file we don't really need it in our code base so we'll delete that dump file we can always run it again so custom rake task to download production Heroku data and load it locally but standard security warning if you have sensitive data in your production database then maybe you don't want to walk around with it on your laptop it is handy there is value in being able to perf test code against real production data so if you want to profile your code or if you want to do some perf testing this is a really good way to do that or just use a staging environment also a good idea so we can do that so we've talked about rake for database management and maintenance and we've talked about rake for just plain old Ruby projects rake can help us make our projects more manageable and make our life more serene that's Lake Champlain in Burlington Vermont one of my favorite places if you have questions I would love to talk with you come up talk to me afterwards I've also nearly finished writing this book, data visualization toolkit rough cut will be available online soon on safari books so if data is or GIS are your thing or if you want them to be your thing then we can talk about that as well thank you