 get through all these. Oh boy. All right. So, my name is Nate. I'm going to talk into the mic while I do this here. And today we're going to talk about mini-test. I know on the program it said, guided read, and we are going to read code today. This is basically a code reading talk. But as I was writing this talk, I realized it's really about pain. And the pain that testing makes us feel. So, we're going to talk about pain. You probably could have titled this talk something else. Mini-test, it hurt me, and I liked it. My name is Nate Berkepeck. I write about full-stack Ruby at performance at NateBerkepeck.com. Performance is my usual wheelhouse. I'm not like a testing guy. I don't usually write about testing. Testing is sort of the last thing on my code skill improvement list sometimes. But mini-test played a huge part in my sort of evolution as a tester and as I'm going to show you as a Rubyist. So, that's why I wanted to come and talk about it here to you today. Wrong way. So, I'm going to start off by telling you a story. I started learning a program in 2010. I was a college senior. Ruby was my first programming language. I wanted to get a job at a big flashy startup. And I thought the best way to do that would be to become a programmer. Turns out that was true. But when I started learning Ruby, I used Michael Hartle's Rails tutorial. I had no computer science background. And if you've ever looked at that, you know Michael Hartle does a great job of teaching you TDD from the start. But he uses RSpec. And for a long time when I started with Ruby, testing was really hard for me. I was just getting started with Ruby. I barely understood what I was doing with Ruby to begin with. And then along comes RSpec. And there's sort of like, well, you got to learn describe and it and all these other kind of words that RSpec concepts that RSpec brings to you. And says you have to learn those too. And for a while that was really frustrating to me. Now I found mini-test. And it was kind of like a brand new day for me. It's like, oh, it's just Ruby. I can just write whatever I want. You might have heard this sort of line before if you've ever done anything with mini-test. I knew the API was very simple and it was a lot easier for me as a beginning Rubyist to write. But I came upon this test and I wanted to stub an object. I wanted to stub an object's instance method. And I was really frustrated because it seemed like there was no way in mini-test to do that. And I went into the mini-test IRC channel and I said, guys, this is dumb. RSpec makes it really easy to do this. And why won't you let me stub an instance method? And Mike Moore, the maintainer of the mini-test Rails gem among other things, I'll just keep going in the wrong direction here, said, well, it's because we already have it in Ruby. It's called a singleton method. I was like, what? You can do that? And that sort of started this long love affair with mini-test where mini-test kept showing me parts of Ruby that I'd never looked at or I'd never understood. And so by reading mini-test source code, we can sort of reengage with Ruby. In this way, pain, feeling this pain of, oh, it's really hard to do X, Y, Z led me to learn something new about Ruby. This is a talk about mini-test. It's in the less code track. So I do have to address the elephant in the room. And that is RSpec. I think the original CFP or something for this conference, this track specifically, said that less code was like 500 lines or less. Mini-test is not that small. But it did, I did get into the less code track anyway. And I think that's because we all understand that mini-test is small in comparison to the other options that we usually reach for when testing in Ruby. But I'm not here to play internet fight. I'm not here to talk about RSpec. I'm here to talk about mini-test. And to get it totally out there, I mean, it's pretty obvious I came to give a talk about mini-test. I am a mini-test guy. And so I want you to take everything I'm saying here with that grain of salt in mind. I don't want you to swallow everything I'm saying here, just wholesale. But this talk is about the philosophy of mini-test revealed through its code. And we can't really talk about that without talking about RSpec. Because a lot of what makes mini-tests philosophy so interesting is what it doesn't do. And we can't talk about what something doesn't do without talking about something that does it. And in that way, I think mini-test is best thought of as a reactionary testing framework. It's a reaction to other kinds of or other styles of testing. Bit of a history lesson here. The original mini-test was 90 lines of code. Ryan Davis sitting right up here in front was nice enough to resurrect that version for me. And it's available there at tinyurl.com slash original mini-test for an interesting read. And if you've ever wanted to know what being terrified feels like, it's talking about someone's library while they're sitting in the front row. And Ryan has said in other conference talks that it was originally a replacement for test unit. He sat down, was saddled with the maintainership of test unit, thought it was too complicated to even understand and wanted to see if he could write something that did what test unit does but in less code. And it's from that beginning that we have mini-tests that we have today. Before we actually read a single line of code, though, I think you can learn a lot from a project's file structure. And it's general where its lines of codes live and how many there are. So before I dig into a library, I think one of the greatest things you can do is open up clock. It's this little library for counting the lines of code in a project. It's smart enough to break things out by file, to break things out by language. So if you have multiple languages in a project, it'll say, oh, well, it's got 10,000 lines of Ruby and 5,000 lines of JavaScript and so on and so on. And if we apply clock to mini-test, you get this kind of like crazy output. But the gist of it is mini-test has 1,699 lines of documentation in the form of its readme and all the non-blank comment lines. And just 1,586 lines of Ruby code. That's also blanks removed. That maybe sounds like a lot, I guess. 1,000 is like a big number. But let's take a look at something else. So RSpec has four major components. Mini-test lives in one gem. There's a huge plug-in infrastructure, but if you gem mini-test, there's no mini-test dash a million other things that get required. RSpec has four major components that get required when you add RSpec to your gem file. That is RSpec core expectations, mocks, and all three of those depend on something called RSpec support. In total, that is 1,843 lines. I just messed up saying that, but it's a lot. It's a big number. It's so big, I can't even say it. Mini-test's entire library is about as big as RSpec's support library. So mini-test is about one-tenth the size of RSpec in total. If mini-test was the mouse in the motorcycle by Beverly Cleary, RSpec is Ulysses by James Joyce. That is a literal relative size comparison. I'm not sure how far that metaphor goes, but Ulysses involves some guy getting plastered, wandering around, and not much happens, which sort of describes my experience reading RSpec in preparation for this talk. So that's mini-test philosophy point number one. Do less with less. You don't go from 15,000 lines of code to 1,500 just by cutting out waste and being a better rubious than everyone else. You get it by cutting scope. So when we go in and start reading mini-test, we're going to be looking for the things that mini-test doesn't do. As another good example, RSpec includes 1,639 lines of formatters just for printing your test results. There's an HTML formatter, JSON formatter, a whole bunch of other ones that I don't know what they do, but mini-test spends its entire library doing what RSpec does just to show you what it did. Another, so moving on from that, I guess, RSpec mocks, and this is a particular example, I think, of a difference in priority between RSpec and mini-test, is 28 times larger than mini-tests included mocking library. You can use any mocking library you want with mini-test, but the one that's included is this mini-test mock.rb file, and that is 28 times smaller than what RSpec includes by default. So without even reading mock.rb or reading RSpec mocks, I think it's fair to say that mini-test is a statist testing library. That mini-test is not in the stalling way, but in the statist mockest testing philosophy dichotomy. If you don't know what I mean by that, I think the late James Golick said it best, a statist tester asserts that a method returns a particular value. A mockest tester asserts that a method triggers a specific set of interactions with the object's dependencies. And this is not... Oh, wrong button. So we know that if mini-test is going to provide us 28 times less mocking power and capacity than RSpec is, mocking and mini-test will be hard. We will experience pain with mini-test, like I did, when using test doubles, mocks, stubs. And I think mini-test is trying to tell us something with that. It's trying to say, don't do that. If this is hard, maybe you should try something else. Maybe you should try code that doesn't need this, that doesn't need mocking. So that was kind of the first instance of, I think, mini-test using pain to put us down a particular testing and coding path. And here's another. The largest file in RSpec core is configuration.rb. That is 758 lines, again roughly half the size of mini-test in total. The largest file in mini-test is mini-test.rb, 446 lines. And what I think mini-test is kind of saying here is that configuration, maybe this is like sort of the old Rails thing that we've always heard, like convention over configuration. I've been trying to run with this pain theme here, so I think it's mini-test knows what's best for you, and you better do it, damn it. Configuration is not a huge component in mini-test when compared to reading RSpec. Mini-test is extensible. And we're going to see that in some of the code that we read here in a second. But we should not expect a huge amount of configuration or that sort of like setting and that sort of business. Ryan has called this turning testing up to 11. I would say it means that mini-test has a testing philosophy and it would like you to adopt that when using it. It's designed to teach you a certain style, and that style is not necessarily up for debate when using mini-test. It's extendable, but it's not really configurable, and I would contrast that with RSpec's approach, which seems to suggest that anything is possible with RSpec. There's all these hooks, there's all these configuration points, ways to make RSpec do whatever you want. And you can do that with mini-test, but maybe it'll be a little more painful than RSpec. And maybe you should think twice before doing that. So you might be thinking, Nate, you've talked for about 15 minutes now and we haven't looked at a single line of code and I agree. So let's talk about mini-test. How does it work? Here's a typical, extremely simple mini-test test. I want to call your attention to the, I guess, sort of concepts or big things that mini-test is going to require you to understand to use mini-test. We require something, okay, cool. Like I'm a Rubyist, I know what requiring does, or maybe I had some idea. There's a class, all right. I've used those before, I've seen that once or twice. It inherits from something called mini-test test. So that's interesting. We need to understand what mini-test test is and what it's going to give us. Because as soon as we inherit from a class, right, we're going to get all of that class's methods. So, you know, mini-test test could have a million methods and now we're working in that world. So who knows what's happening there? And then we have a method, which is test the truth. I could make a bigger example here, but if you had a larger mini-test test, you would notice that every method or most of the methods started with test underscore something. So that's interesting. Probably has a meaning that we should look for in the code. Finally, we've got this method, which probably, because we can guess, there's an inheritance here, assert equal probably comes from mini-test test, which takes two parameters. So those are the concepts that we're going to be looking for digging into the code. If we run that file, just with Ruby, the command line tool, we get all this output. We don't know how that output shows up. So let's start from the beginning. Requiring mini-test slash auto-run. My guess would be that I should go looking in mini-test library for a mini-test, or for an auto-run.rb file in lib mini-test auto-run, which is exactly what I find. Slightly simplified, I cut some stuff out of the beginning here, but this is what that file looks like. It just requires some other files, which is great. We know how that works, and calls mini-test.auto-run. Well, where does mini-test.auto-run live? It lives in mini-test.rb, which was what we might expect if we were looking for a class method on mini-test. And it looks like this. Now you might be thinking, Nate, I thought you were trying to sell me on mini-test was the simple testing library, and look at how complicated that method is. And I think that's interesting. I had that same initial reaction looking at this code, and I want you to pay attention to the shape of this method. Like if you stood here and squinted, and just paid attention to, all right, it's about 10 lines, it's fairly sequential. There's one sort of long line here, but otherwise it's pretty short. And there's just one method here, right? I think, yeah, like there's only one other mini-test method that's called here, mini-test.run. We've got this some Ruby stuff here, like add-exit, we're setting some variables and stuff, but there's only mini-test.run. That pattern is going to happen a lot more as we keep reading this library. And I think that that pattern is in great contrast to our spec, which I'm going to get to in a second. All mini-test is doing here is installing an add-exit hook. If you were wondering, from the previous example, how does mini-test know to run our tests? It's required before we defined any of our tests, so how does mini-test know where our tests are and how to run them? That's because when your program runs, there's nothing to do, because there's no code to execute. There's no mini-test.run at the bottom of your test file, so your program just exits. And when it exits, basically we do some other stuff and then call mini-test.run. Nine-tenths of this method is defining some stuff to give you the proper Unix exit code, but really the meat of it here is mini-test.run. If we just sort of collapsed the add-exit block here, it would just say add-exit, do some stuff, unless class variable, install that exit, then set it to true. Clearly just meant to prevent us from running that method twice. Fine, not complicated. This is how our spec does that. Not that much more code, maybe twice as much code, literally just does the same thing, installs an add-exit hook. But again, I want you to pay attention to the shape, very different, very interesting. One, two, three, four, five, and I eliminated five other methods that our spec defines to carry out this whole add-exit process. This is interesting, you probably can't read this down here, but the disable auto-run class method on our spec just sets the auto-run disabled instance variable to true. That's an interesting pattern that you don't see anywhere in mini-test. I think it's a very interesting expression of this is what our spec considers readable versus what mini-test considers readable. Mini-test says, you're Rubyist, you know what this means. It means that we've installed something at exit. Our spec says, well, we need to put that behind this accessor that we've written here. Very interesting, you're going to see that a lot. It's also kind of funny. Our spec does all this to install this add-exit hook and then tells you not to use it because they want you to use the rspec test runner. When you type rspec whatever file dot rb, there is no test runner included in mini-test. You just use the Ruby command line interface, although there are plenty of mini-test compatible test runners like the one you use in Rails. That's the better view of that example I just gave you there. Again, very interesting differences in opinion on what is readable and what is not. I think that this means that mini-test is trying to tell you that mini-test thinks that you should do the simplest thing that could possibly work. I want you to note the differing philosophies here on simple. rspec, in terms of its testing interface as well, tends to think that simple is English readable or looks vaguely like English. That's sort of the whole spec syntax mindset, but I think in a lot of ways mini-test says well you're a Rubyist. Ruby is simple, just read Ruby. At the end of that add-exit hook that we installed earlier, you'll remember we called mini-test.run and that's what determines the exit code. Let's look at mini-test.run. Alright, so right off the bat before you even get to, again look at the method shape here. Very different than a lot of what you find in rspec. I'm kind of picking out some of the most complicated stuff in mini-test like this whole stack of, mini-test goes to this whole stack of methods before it actually runs your test and it's kind of the more complicated stuff in mini-test. There's a lot that's a lot simpler than this. But right off the bat before we even get to the second line here, check this out, self.run, reporter options, where the hell are the parentheses here, Ryan? Where'd those go? So let's talk about Seattle style for a little bit. For the uninitiated, Seattle style I think has a lot of different definitions for the purposes of this talk. I'm just going to say it's sometimes interpreted as don't use parentheses and method definitions. Sometimes you go further and say don't use parentheses except where absolutely necessary. Which kind of leads to some interesting style choices. It's not super popular. David Brady gave this great example of parentheses usage. I'm going to preface this by saying this sort of code that you're looking at right now does not exist in mini-test. This does not something that you'll, similar that you would see in mini-test. I'm just making this up. We've got hash.fetch delete and then puts. I agree that parentheses makes this more readable. Parentheses are sort of like this tunnel vision. You can say what's the innermost parentheses here? The foo symbol. Let's go out one level. Hash.fetch. So we're going to fetch the foo symbol from this hash outside of the next set of parentheses. Array.delete. We're going to delete that thing from an array. Got it. And then we're going to put it to the screen. Okay, fine. But maybe that's not the dichotomy here. Maybe that's not the reason that we're not using parentheses. What if that code looked like this? This is a contrived example, so I don't think it plays as well, but I couldn't think of a good actual code example for this. What if we blew the original code out into three lines instead? Did something like actually describe what the thing was that we were getting out of this hash and then putting the side effect onto a different line? Just an idea. I think this is a lot more readable. I think it might be interesting. I honestly never considered this until I read the mini test source code all the way through. It might be interesting to take parentheses out of your code when you're writing it and then see what that makes you do. What does your Ruby code have to look like when not using parentheses in order to be readable? Is that result more readable than using parentheses at all? An interesting exercise. I know I'm going to be doing that in the future just to see if I like it. What I'm trying to say with all this is that I think mini test has a philosophy that pain is good. We can navigate our coding with pain. We can use it as a signal. I feel pain here. What does that mean? What should I do? Is there a way I can make this go away without just hiding it? I already talked about the mox example. I think what mini test is saying there is mox are painful here. Try writing code that doesn't need the mox rather than coming up with a better mocking library or a mocking library that makes this code easier. Mini test also makes adding assertions very easy which I'm going to talk about in a second. When it's doing this I think it's saying it's trying to give you the opposite. It's saying well assert making new assertions are easy in mini test so I am encouraging you to write your own assertions. RSpec makes it easy but it makes it difficult to understand. Seattle style can encourage the use of local variables, intention revealing method names and perhaps I think the Seattle style tries to suggest to us parentheses are sometimes a bandaid on a problem that we should fix without them. So I've kind of outlined how mini test starts running your tests. How does it actually know where your tests are? How does it know how to run them? Mini test has this class runnable which mini test test inherits from this, runnable, and basically I've omitted a ton of code from runnable here but all it does is keeps track of this class variable runnables, simple enough. And then you see this method class method inherited which just adds something to the runnables array and then calls super. I'd never seen this before so I had to go look it up. I like first searched through the project and said well where is inherited defined? It's not in the project. Okay well must be something in Ruby. Inherited is a callback that you get for free. I think it's on class might be on module and it's just called anytime that you get a new subclass defined. This is just straight from the Ruby documentation as an example. If we subclass foo this method will get called with the new subclass as an argument. So literally all that's going to happen here is my test will get passed as an argument to mini test runnable inherited which will add it to the runnables array. Eventually mini test looks at the mini test runnables array for all of the test classes that it knows about and just runs through them one after another. I'm not going to talk about test discovery in our spec I ran out of time. Standard lib is used very heavily in the mini test source code. We see things like option parser, thread, mutex, string.io, tempfile and in something I can't say I've ever really seen before mini test will use your system diff tool when printing the difference between two large objects like if you assert equal some huge array some other huge array and like one thing and that huge array is different. Mini test will use diff to figure out which thing was different. RSpec implements their own version. RSpec implements that in Ruby and mini test instead says well I'm just going to use what's provided. Interestingly a lot of RSpec re-implements a lot of core and standard lib classes. They have their own version of set, they have their own version of flat map, they use their own version of thread local variables and they even have their own RAN implementation. Some of these I understand. These two I think have Ruby version issues. Not sure about thread local variables but threading is a total you know put your hands up thing in Ruby so maybe there's a good reason for that. RAND I don't know. I was looking at I didn't think I put the example in here but RSpec uses its own algorithm for setting the random seed in your test suite. There was some code comment that this implementation that they use is like just somehow better. That's it, it's better somehow. I don't know. So I think mini test has a philosophy to use what is given to you. To use what is given to you by the system, to use what is given to you by Ruby itself. That's an interesting philosophy that I think is is something that we're really not used to as Ruby developers and I think it's to our detriment. A lot of us have a kitchen sink philosophy when it comes to our gem files. If there's some library that will save me 10 minutes of effort that I can put in my gem file then I will do it. Devise I think is kind of the worst offender here, not because devise is a bad library but because we use devise to do simple user pass name authentication that we could just write ourselves or use Rails facilities for that and be done in 10 lines of code anyway. So I think that's an interesting philosophy point from that we can take out of reading mini test. So how do we actually define a, am I saying test here? Yeah, so how do we define a test? How do we define the, if we say a class file is sort of a collection of many many different things we want to test in mini test. How do we define one of those? This is in, where is this, mini test test, test.rb and it gets called in mini test.run which is on a previous slide. We ask a mini test test subclass for its runnable methods. We say dear subclass please tell me which methods you would like me to run. On mini test test that is really, this is all just ordering them. It's just this methods matching some regex which is just matching tests underscore. The methods matching method is literally just a one liner that basically just calls grep on the collection of methods and brings them back as an array of strings. So really it's just okay well a test in a test in mini test is just a method that starts with test underscore. Also interesting I think that this is not configurable. There's no, you could monkey patch it of course but it's not like some, there's not some hook provided that lets you change out that regular expression. Very different in RSpec. RSpec has from what I can tell 13 different ways to define a test or an example. Granted they do a little different things in RSpec but there are three that do exactly the same thing example in specify. There's ways to define a test while also focusing and skipping at the same time and there's also a pending example. Pending apparently has some other semantic meeting in RSpec and this method here is also provided for you in RSpec so you can do this yourself. You can say well I want the smiley face emoji to define a test method in my RSpec test so I can do that. Mini test says that doesn't matter just do it this way it's fine. There's maybe some reasons for this but if you don't think that exit being a way to skip a test because of I don't know I think J unit or X unit used to do that. If you that doesn't matter to you then maybe RSpec is providing other things that don't matter to you. This is the RSpec implementation of define example method does this thing called item potently define singleton method. I'm not a computer scientist so like I don't really know what this word means. I know what a singleton method is but I don't know why they didn't just do that instead of calling this other method that I guess does it for me. There's some metadata I don't know what metadata is and well this I understand examples. Yep there's some examples here we're adding an example there that's cool. Very different approach. So what I think mini test is espousing here is to reduce our API services. Let the user define their own aliases if they want and make it simple to do so and I think that kind of goes along with limiting configurability. I'm kind of running out of time here. I'm okay. Yeah just just talk faster Nate. Let's talk about assertions really quick. This is an example of assertion in mini test. The structure as you can see is really simple. If you look at minitests assertions.rb they literally all look like this they're all like three lines. It defines a message that we print if the assertion fails and then makes an assertion using the assert method. So minitests doesn't really provide any facility for writing your own assertions. It just says do this on your own tests. Just open up minitest tests or open up your test class and write something that looks like this. There's no specific facility like what our spec provides to define a new assertion or matcher as our spec calls them. This is what assert looks like. Not any more complicated. We increment our assertions count by one and then run the test. A thing that we want to test. If that returns true we just keep moving and if it doesn't return true we're going to raise this special kind of exception which minitest handles in its own way. I don't have time to get into that unfortunately. And you might have noticed here that message is a proc which is sort of interesting. Maybe a little more complicated. Originally it was just a string but if you think about it if we're calculating the error message every time we run the test eventually that's going to get very expensive. So by putting into a proc we can only calculate an error message when we actually need it. You don't have to do that when writing your own assertions because if you if it's just a string or whatever minitest will just print it. If you don't care about that or if it's not a big problem for you. And the way that these get included into your test these assertions is we just include the assertions module in minitest test. We've all done that a hundred times. No big deal. Our spec I honestly have no idea how they do this. The expectations gem itself is like 3,000 lines and then there should have matchers which is another 6,000 lines. I just I really honestly I've sat down and read our spec to try to give it an honest comparison in this talk. This part of our spec I could not understand. This is the facility that our spec provides for defining your own matchers. It's interesting because I think this is very characteristic of our spec. It reads quite well if you like read it out loud in English. But these are all sort of new concepts which you have to go learn about and understand how they work. This define method where this match method comes from I have no idea and what these block arguments are actually doing I think are all questions that you have to go ask our spec about. Whereas this use a module and call this method that raises a minitest assertion. That's pretty simple to me I guess. What I think minitest is saying is that abstraction is the enemy. The pain here is that you will have to write your own abstractions sometimes. I think that's okay. I think that makes us better Rubyists and I think learning to write abstractions well is one of the greatest skills that we can learn as programmers. And God forbid that you learn some Ruby along the way rather than learning how to better use our spec. To extend and use minitest in a greater capacity you usually just have to write more Ruby. With our spec you have to learn to use our spec better. Because it's a DSL that you have to interact with. So I think I hope this is shown through a little bit of code. Though not as much as I would like that minitest in a lot of ways is a philosophy about pain. At least it is to me. And maybe you've had pain with minitest in the past. Maybe if you think back and on a time that you've had struggle with minitest or anytime you've struggled with Ruby really did you just give up and go through devise in your gem file or go through whatever gem you thought you needed to get this done without you having God forbid having to write any more code. Or did you listen to that pain and did you think about what that pain was trying to tell you? I think in programming like life what doesn't kill us makes us just that little bit stronger. Thank you that has been my talk. Please go read minitest if you haven't it's really easy to read and go test more. And my slides are available at that URL. I guess I have five minutes so if anyone has any questions? No? Cool. Thank you for listening to me gap 45 minutes. Appreciate it.