 Hey everybody, my name is Yehuda Katz. I'm here to speak about writing code that doesn't suck Some other interesting facts I work for engine yard engineers awesome place to work There wouldn't really be a murb without engine yard or some other cool things we're working on But I'm especially gratified that I've had a good opportunity for the past year or so to work on murb for big chunk of That time and I think it really shows when you have the opportunity to to work on something The difference between sort of the scattered approach to putting something together and being able to focus hard on it Coming up with a vision and being able to drive it forward really matters. So Thank you engine yard for that Engine yard also does Rubinius and another project called vertebra that we're sort of working on it's been announced. So Yeah engineers cool. Hooray So That wasn't a sarcastic hooray. It was a transitional Transitional hooray so I Don't really have to sell people on tests here Ruby land Because there's the path that we've been sold and that I think rails really Taught us, but it's really a path that we've been That we've adopted and embraced as a Ruby community And we think that this that there's a solution here for writing good code code that doesn't suck So usually goes like this write good tests and there's usually like five or six talks at every one of these conferences about what good tests mean So immediately We have a path, but we actually don't really agree about what the path is So and people vehemently disagree about the path is But step one is write good tests, right? Step two is have a CI suite So make sure your code doesn't break and then profit, right? So usually that's That's the story. That's what we're told. We're told tests are important We all say yes, we agree tests are important Everybody says write good tests and people feel like they write good tests and we have talks about how to use our cuff to show that you're covering your code Very few talks about what entails a good test, but a lot of talks that tell you you should write tests And we agree, but then a bunch of things happen Happen where people Backlash they say testing is horrible, right? So this is a recent posts video Hampton said unit testing code often contains more bugs than its non unit testing counterpart to the guy wrote Hamel And then there was a testing is overrated talk, which actually doesn't say testing is bad But it says that unit testing is just one part of a larger Picture and they also said unit testing is overrated doesn't really work doesn't actually solve your problem that you want to solve So what's going on here? We all know unit testing is awesome We should do it everyone thinks it's the best thing in the world and solves all our problems And we won't write buggy code anymore if we just unit test and then there are some high-profile people that say no not actually useful I Person Pampton says I personally have experienced large amounts of fail when I tried to unit test and in fact Pampton says He fit he finds that his code that doesn't have any unit test is more likely to not have bugs Primarily because he doesn't rely upon what he considers to be a flawed measure. So what's going on? So let's take a step back and review sort of What we know about testing and why we know what we know the answer is we came from rails so Yesterday mass asked everybody to raise their hand if they came to read from rails I'll reiterate the question raise your hand if you originally the first ruby that you did was rails right everybody right so Rails actually had a really strong testing ethic from day one rails was one of the early frameworks That's actually built-in testing as a primary thing that you should do with your code when you write rails code the thing is that rails didn't really ever say what a good test was and people sort of looked at rails as generated tests and What other people were doing with rails tests and sort of adopted that as that is what a good test means What people do for rails tests? But but at no point in the that I am aware of in the ruby community was there ever a good vigorous debate About what sort of test is the right sort of test and how you should test right? There's just testing is good rails says testing is good. So we all laugh at the Whatever other language where they have a 10% testing rate or whatever we think we're superior But we never really analyze this for real With that I'm going to move on to some things that rails does So I'm not going to talk all that much about rails as you probably saw in the blurb This talk is actually about how murb used to really suck But there's some things that rails does that really sucks in terms of design that I can't murb never did Even though we really suck pretty bad Earlier, there's some things that even we didn't do so I have to dig into rails to show the point that I'm trying to show And the first thing is this alias method chain thing and People have heard me say this before but I'll just I looked around for a mailing list post that sort of Made clear what was going on and there was an early mailing list post in 2006 that said If you're unfamiliar with this method read it you might never use it since it's internal to rails But study nevertheless to sharpen your ruby mind and people probably have heard me say alias method chain sucks and probably don't use it But probably sick of hearing me talk about alias method chain. So why do I always talk about alias method chain? Why do I think it's so bad and the answer is it's an exemplar for a pattern that rails uses that makes the Writing good code really hard and that it's an exemplar for ruby as an API We have an awesome language in ruby ruby is really great language. It lets you extend modify Dynamically do metaprogramming you can do a lot of stuff at runtime, which means you can modify extend Use abuse other people's code and a lot of people sit think that since ruby is so good at manipulating other people's code You should just use ruby as an API. In fact, that's sort of rails is philosophy They say we why do you need an API? Just alias method chain whatever you want to extend We don't need to provide you with a hook, right? We don't need a hook We don't need to tell you what methods are meant to be used and what methods are not meant to be reused I think the best I've ever heard is if it's not no docked it's public Yeah So This philosophy is ruby is an API and in fact it works pretty good When you have some code and you're modifying it and You have a small amount of it, but it really defies maintainability and code reuse so When you have this big tangled mass of everybody alias method chaining everybody else's code and no defined notion of where You should hook into somebody else's code you end up with a tangled mass of things that work well But a tangled mass of things that as the whole become very hard to maintain and near impossible to reuse And and that was my experience doing rails in general and part of why I originally moved to MIRB was this problem of As the community got larger and there were more and more things modifying how rails worked The chance of collision conflicts this plug-in doesn't work with that plug-in Got higher and I think it's only gotten I haven't used rails really for anything significant in about a year but I think that problem has gotten worse from what I hear in terms of Having to magically know what versions of plug-ins work with other versions of plug-ins what versions of plug-ins work with rails And so tangled masses of things are not good And so alias method chain is an exemplar for that problem of just use ruby as an API We'll sort it out ruby is very good. It lets you modify things at runtime so I'm gonna talk about a lot every so often I say this and I get some objections So let's talk about why people want to use alias method chain So the argument goes ruby is inherently dynamic and it doesn't have interfaces on purpose, right? If you ask Matt, he'll tell you I don't want interfaces in the language. I think it's a bad idea and So why not? Why make my code Java sort of the the small version of the argument? Why are you telling me that I have to turn my wonderful dynamic extensible ruby code into this static Interfacy Java thing And rails embraces this philosophy of dynamic is good Interfaces are bad They have alias method chain you everybody extends native think types directly. It's very common to do Active record base thought send include some modules, so you're extending active record base directly There's no notion of when certain extensions might be dangerous and no notion of what's likely to change so Because of the fact that it's ruby people who like alias method chain embrace this fact they say What we're just gonna change whatever we want anybody can change it You just have to deal with this you have to deal with the fact that it's a dynamic language And there's no there's not just not gonna be any interfaces And there's also no notion of what's considered to be private to the library because the argument goes Why should there be anything private? Why shouldn't you be able to just call things? And I remember a year or two ago. There was a strong stream strain of thought that said that The private and protected are actually just hints They're not really meant to be real and so just don't private or protect anything Everything is public because it's ruby right So where where does this break down? Where does this philosophy break down? The first place where it breaks down is modularity. You can't easily modularize your code if there's no known Places to plug in so you could start modularizing your code, but it breaks down very quickly when you start to do it We learned this very quickly when we started with murbs 0.9 and we decided that we wanted all the pieces of code Sort of like the way Unix works with have small things that are good at what they do best and you could hook them together We wanted modular code and what we discovered very quickly was that if we didn't have extremely well-defined entry points into Each module it became nightmarish to maintain 20 modules, right? So modularity is the first place where this notion of Ruby as an API breaks down The second place where it breaks down is future-proofing The people who talk about Ruby as a dynamic language are really talk for coming from the perspective of rapid prototyping and development But they're not they're completely ignoring the fact that when you do that you are unable to look into the future and Write code that will work against a future version of whatever code that you're writing against and I would say not only do they not It's not that they don't deal with the problem They actually embrace the fact that that isn't true right rubies a dynamic language. It's a good thing So that's for libraries for web apps what I mean by future proof is that When you write maintainable code You only use extension points in your own code that you're personally committed to maintaining right so you make sure that your models have extension points that are That you're going to continue to maintain in your controller and you don't just use any model You don't just expose everything as an adder-accessor, right? You only make you only expose a few extension points that you're personally committed to maintaining in the future and use those in other parts of your app Perhaps the biggest problem of alias method change is just that it's bad design, right? So I'm going to read a little a short quote from interface oriented design the book and This is from the introduction designs that emphasize interfaces are loosely coupled and that's a good thing If you have only an interface to which the code you cannot write code depending on implementation, which keeps us honest And in fact everywhere other than Ruby and every other language. It's sort of accepted that this is true That you should write you should good design entails Thinking about and designing the interfaces that you're building by bad design what I really mean is coupling and Coupling violates something called the single responsibility principle Which basically means that you have objects that are good at one thing that talk to each other through small interfaces The most obvious example of this again is Unix piping where you have a very small simple apps that know how to do very simple things They have one extension point the pipe Right and yet they can do all sorts of wonderful things and you don't have to worry about those extension points breaking because there's only one of them Wonderful future proof all you have to do is design an app that reads standard in it works Right, so single responsibility principle sing sing simple extension points These are actually really good things for good design when you don't have these things you end up with really bad design sort of accepted as how it is So just gets back to the tangled wires problem, right? You want to design something that doesn't end up in a heap of scrambled tangled wires So What do we do first thing to do is read this book This book is I feel like I'm being Giles here who came to large numbers of books But when I first started thinking about interface oriented design about a year or so ago when we started working on MIRB I hadn't really I'm not a CS person I read a lot of books, but I so I never really I don't know what sort of strains of thought there are out there and I started sort of cohesivizing the idea and I discovered that there was a book that's pretty old in terms of Ruby pragmatic programmer world. It's One of their first books and it goes through step-by-step what interface oriented design means what sort of this talk is about Step one is you interface with the outside world in a library. That's your API So you're gonna write some way that your library interacts with the outside world. That's what I mean by interfaces in a web app That's your request response cycle, right? You're the way that your app interacts with the outside world is through the requests and responses It's possible you have it's possible you have other internal interfaces And you probably want to design these also in accordance with interface oriented design But I'm not really talking about these interfaces today. I probably will have to deal with these in question-answer period I'm talking about external interfaces today the Interfaces that you expose the outside world and that other people come to rely on The reason that I care about these is that those these interfaces are the ones that if you break other people will care and It's possible that in your own app You you can segment out your own app so that some part of your app cares about the interface of another part of your app And that's I would consider that also an external dependence external interface What I don't consider an external interface is like your helpers right because The only thing that cares about your helpers are your views if you split a helper into two helpers Or you move a helper in line that doesn't actually modify the interface if you did that know nothing broke right if you take a helper and break it into two separate helpers and The app still works perfectly fine. That doesn't break anything. So while there may possibly be some internal interface that may have broken I'm not really talking about those interfaces today And I don't think that regression suites actually care about these and we'll get back to that in a moment Which brings us back to testing so Hampton said Unit tested code often contains more bugs than it's non unit testing counterpart I've an alternative hypothesis Unit tests are not regression tests And by that I mean unit tests of internal code are not regression tests What am I saying here? What I'm saying is that a lot of people when they come to abandon testing They've spent a lot of time unit testing Internal methods and have no tests or very few tests for the thing that they actually care if it breaks Right. So in MIRB we care of render works We care that if you have a content type defined as HTML and that you ask for HTML HTML comes out We don't quite as much care whether the method that figures out what the content type is actually works the same way It worked yesterday We do care that the stuff that that produces Namely that when you ask for HTML you get back HTML and all the edge cases that surround that we care that that works And so a lot of people end up writing unit tests and Hampton. I presume is one of these people and they test that The method that looks up HTML works and they've completely missed out the thing that they care about And so they have unit tests. They feel really confident about the unit test They feel like they've correctly unit tested the method that looks up HTML But then some other part of the app breaks what they care about And so they start feeling like this warm blanket of unit tests isn't really the warm blanket that The path says it is and they start saying it's better if I just test everything I'll spend all my time banging on it because at least then I know it works When unit tests burn people, right? It's not to say unit tests are bad. I'll get into that So I said that the talk is about code. So let's look at some examples When we moved from merbo five to merbo nine, we didn't want to change the API So this brief history of Merb is that Merb was little hack then merbo three was Very tiny rails merbo four was some extra features and then merbo five was a fully feature rich API But when we look when we started looking at merbo nine, which is actually when I came to engineering in the first place We realized that there was a whole bunch of stuff in merbo five that really could use a rewrite And we were obeying the principle of write it once and throw it away We wanted to start from scratch and we did that is in fact what we did We started with a blank repo and started over and we said hey I bet you some of these tests would be useful because we want to keep the API So we'll just copy over these tests. What do these tests look like? So we had stuff like this right and This is a common merbo five tests It's probably not as egregious as some of the tests you've written But I think it's a calm its common practice do something kind of like this Okay, you say you make a simple responder and then you stub a bunch of stuff to pretend It is the method that is being used internally the exact method because the method that Negotiates content type calls provided formats and returns an array with with X XML and HTML But we were throwing it all and starting over who knows if provided formats is still what we want We don't care about this test And we had tests after the before filter that did stuff like this We would say it should respond to perform content negotiation, right? So this stop is a serious offender. We're here. We're testing Whether perform content negotiation a private method exists at all that doesn't seem like a very useful test at all this over here this stub params and return empty hash is not portable to o9 because We don't actually call perform content negotiation anymore, right our methods work differently This is an assertion. We're not committed to and on and on, right? Basically, we had a whole bunch of this sort of test and no test that tested what we cared about, right? We had Gads and gads of tests that tested internal functionality and no test that said if you ask for HTML you get back HTML zero tests and some people have 90% of their tests are performed content negotiation and 10% of them are tests to make sure that when you ask for HTML You get back HTML, but the bottom line is that those tests these tests? Can't work for refactor. They're not useful, right? You have to throw them away if you actually refactor your code at all None of this stuff works anymore and now you have none of this stuff works anymore You have no test that confirmed that the stuff that you actually care about still works, right? So this is not to say unit tests are a bad thing, right? What it is to say is that we actually care about having tests so that when we refactor we can confirm that stuff still works now Right, we need a lot of those sort of tests the ones that confirm that the stuff we care about still works So what's the public API? It's if you ask for HTML you get back HTML. So what does merbo 9 do it says? Go to the HTML default controller index action. The body should be HTML default, right? Unfortunately, there is a few Non-public items here address that shortly, but it's only a few things for the entire suite the entire suite uses these two private ish things Of course since we're doing a real thing as opposed to testing some micro thing We actually need a little bit of setup, but the setup is this right? It's really simple We're actually moving to something like this Where you say request slash html default should have body html colon default and if you've used merbo you recognize this This is the exact same kind of test that you use in your merb apps which goes through the entire rack So no now we no longer have to rely on private interfaces at all We rely upon the interface that we use to talk back to rack and now we have Tests that cannot break no matter what if we don't change the API so no matter what we do internally We can't break these tests Here's another Thing that we used to do oh Here's another thing that we do. Here's another example of a test that we do is Controller equals dispatch right we say htp accept application XML and then we say controller body should equal XML class provides equals true and that would probably be converted into a test like this So we're sort of moving towards these converting our test over into these The nice thing about the sort of the non awesome test like I said though is that It's even though it's not awesome in that it uses a couple private things if we do change the private Interface we only have one thing to change and our test suite will work right the dispatch helper is just a test helper So we just change it once and everything still works And then that's some setup Another example the o5 responder spec so first of all After every part of the o5 responder spec we did reset default mime types I don't think that exists anymore in o9, but even if it does exist. It's not a private AP. It's not a public API This is also emblematic of Weird bad testing which is that you're forced to clean up things that the API doesn't actually allow you to clean up so You have a bunch of stuff you're doing and you need to go clean up things that actually you can't you can't unstart murb Can't do it right once you start murb. It started and we do not maybe one day We'll let you unstart murb, but today we don't So what a lot of people will do is they'll do a massive test thing where they'll try to figure out all the side effects of Starting murb and undo them the problem is that if other side effects happen Because we change how murb works you have your test no longer correctly unclean up murb. So you shouldn't use Private apis or magic apis that you invent for your tests just to clean up things that you can't clean up you what we do in murb, and this is perhaps It's it's a little bit messier than the way people normally run tests, but it's still It still works perfectly is that we run every test every spec file and its own its own process so Since we start murb a lot in our tests. We don't want to have to unstart murb We just start another process and we have a trick that we use with forking to make it not slow So Basically the reason that that would normally be really slow is that you have to actually load a bunch of files over and over Right so and the benefit that you get from loading it all in one process Is that those flat file load just happens once and you reload all you load the files again again again and again? Right what we do is that we fork right after we load all the files But before we actually do anything And then we jump out of the fork when we're done and start a new one So we still get the benefit of isolation basically sandboxing for each of our specs without the speed hit that would happen The bottom line here though is that I think we're probably gonna We are going to Probably package that up as a gem or expose it in some way so that people can use that same technique for their own For their own specs, but it's not very complicated and it's really good The bottom line is that if you you if you rely on clean up bad things happen We did that a lot in 05 and it just made our specs very un-maintainable So we had specs like this should respond to add my type. These are not useful specs, right? It's not actually useful to write a spec that says that some method Exists on Merb Right. I've seen this more than once I sort of hesitated about including it here because it's really egregiously crazy, but I know people do this So this is not useful because somewhere else if you have a spec that actually calls available mind type It will confirm that available mind types exists. You don't have to have a spec that says this method exists It's not testing anything useful Then we have it should give access to the available mind types We say Merb that available mind type should equal this thing here. What is that saying? It's saying that Merb that available mind types is this internal thing, which is how it's implemented, right? And actually that let us check to make sure that what we were doing worked But it actually just checks to make sure some internal implementation details, right? And if we change that which of course we did in 09, right? We just we didn't end up using Merb colon colon responder mixing colon colon rest colon colon Capital types as the the storage mechanism and in fact we we ended up doing a lot more complicated things with storage of mind types We have two-way hashes and whatnot. So basically this test isn't gonna work. We can't port this into online It doesn't actually test the thing we care about Right who cares so what's the public API? What do we actually care about? We care about the fact that when you add a mind type it makes it available to controllers So here's what the 09 Spec looks like we say Merb that add mime type some random mime type that doesn't exist and we use the API that this is one of many specs Which we say that it Takes application foo and it has this special car set right and then we go and we Request and I'm just saying better way because this is the better way that we're moving to right So we say response equals request and then a controller that we set up And then we just check to make sure that the headers of content type are the appropriate thing that it's application foo And it contains the car set that we said in my mind type This is actually the thing that we care about this is the thing that we want to make sure still works tomorrow, right? this over here Not so much what I care about Tomorrow if this breaks, I don't care right if this breaks I actually care the problem is that I can care all I want, but if I don't have a test like this It's not gonna actually be caught If I still store it in colon colon capital types I'm not gonna detect the error even if I can no longer get to it from a controller and Another example here. We have to set up a Simple scaffold the reason I'm showing these setups is that you might assume that it would require significant amounts of effort to set up These complicated steps That's usually what people Raises an objection when I talk about this and it just turns out to not be true turns out that the amount of setup that you Have to do in well-factored code is Minimal right because you're basically testing a real interface and if MIRB was really complicated to use people would not use it right so This is all you actually need to do a controller in real MIRB So it's all you need to do a controller in a test. It's possible that rails might make it significantly harder. Don't know So the last example here we decided to refactor the dispatcher so the dispatcher in 0.5 look like this I Have to fit it on two pages. It's a lot of code one one method big method and We decided that since most of the work of the dispatcher actually entailed dispatching a request and actually This is a good sign if you have a lot of static methods that have as the first parameter the same thing That should be a good evidence that what you really mean to do is make it methods on that thing, right? That's sort of the CC plus plus shift and so similarly if you have static if you have a static We don't call it static in Ruby. I don't know what we I don't think we have a name for it We have class methods So if we have a class that only contains class methods, right? And all the class methods take as their first parameter the same thing. That's evidence that they should not be static methods So we decided to move it to the request so we did this right now it's now request does handle and The request handle is still pretty small right fits on one screen in relatively large font so but We didn't actually want to break the API a lot of people actually want their request to dispatch correctly Merb we assumed people wanted that So we wanted to make sure that this still worked and had we used the o5 tests We couldn't have possibly confirmed that everything still worked, right? So we removed the request that handles some examples We decided to use to we moved find route into the method into the request So now instead of inlining the thing that finds the route we have a method called find route that does the work for us and Similarly, we have a method called controller previously There was a big chunk of code there that detected what the appropriate controller was turned out the request actually has all the information It needs it doesn't even have any local Variables here, so we moved it into the request, right? turns out We actually could not have used the o5 test We had to build before we did this this dispatcher suite We already had some tests, but we have to go back and do a Tremendous amount of work to test a bunch of edge of edge cases in a dispatcher Through the code that people care about right what what do you care about the dispatcher you care about the fact that When you make a request it actually gets dispatched to the right place There's an error it raises an error page in Merb. There's a whole bunch of special error handling We have to test that right, but you don't actually care about what how finds route exclamation point works or Whether in fact it goes to Merb dispatcher dot handle or some other method request dot handle none of this matters and that's sort of The point so to summarize There's basically three rules of testing that I personally adhere to Number one if you have a broken interface at least one test will fail Rule number two if you have a working test zero tests will fail Now this is hard But we want to get to the point where this is true and the third rule is less a rule and more a Corollary, but it write tests about what you care about right write more tests Testing that the public interfaces of your application or your library work and less tests about how the internals of your implementation work So at this point I usually get splattering of objections that I'm crazy and it's really hard or it's slow and Turns out that it is slow naively, but that is possible to make it fast. So It's slow first of all the Merb test suite runs in I think 45 to 60 seconds for like 1300 expectations So doesn't need to be slow. So this is a Merb test suite that does everything full stack 1300 expectations of full stack in 45 seconds. So doesn't need to be slow Our forking trick makes it faster, but my response to it's too slow is okay. Let's make it faster, right? We can figure out how to make these tests run faster It's too slow is not an excuse to not have tests and that's really the point, right? If you don't if you only have tests that test unit stuff You don't have any tests that confirm that your code still works Merb 05 had no test that we were able to port over zero tests Even though we had a lot of expectations and we felt like there was a nice security blanket So it's too slow is not a good excuse to not write tests It's the same objection that people make to writing tests in the first place Right, so we need to make it faster if it was extremely slow if it took an hour Then we could like have a discussion about it What I find is that it takes some extra amount of time that a is worth it and B can be optimized And I think Merb proves that it's optimized The second thing is it's too hard and again, this is an objection that is raised against testing Right, but we don't actually believe that hard Testing being more hard than not writing tests is a good argument because we believe that having an automated way of confirming that your Code doesn't break is worth the difficulty, right? We already believe that so The fact that it might be harder to test your code through the interface is not a good argument I personally find that once you get into the groove of writing tests that test what you care about You find that that's sort of tests that I showed earlier where you basically have a test that just tests the real thing that you care about It's relatively easy in terms of applications, obviously this requires something like web rat and Merb has been Doing a bunch of work to make sort of acceptance tests that I'm advocating here easy So we integrate directly with web rats. You can do things like You visit some URL click link fill in these fields click submit button, right all on your tests And so it's too hard is a if it's that's nice that it's too hard But you actually want to have tests and the other things aren't actually regression tests and B We can also work to optimize how hard it is Merb has been I think on the forefront of optimizing that We have we've been working directly with web rat including at this conference to finally release something That's really a good integrated solution Another argument is that unit tests are useful and that is in fact true unit tests are very useful They just are regression tests Unit tests are very useful for isolation when you find a failure being able to figure out where it is They're very useful for design. So I personally find that writing unit tests as I'm designing something helps me think out the problem I think other people find that they think out the problem better without unit tests I just disagree I find it easier to use unit tests But they just aren't regression tests, right those tests which are effectively what I showed earlier Possibly not as egregious, but roughly that just aren't regression tests they don't confirm that the things you care about still work and You should be building regression tests that work. You can also build unit tests I feel like I have to say this at every time I ever give a talk about this I am not saying you should stop writing unit tests I'm saying that you should swing the focus towards writing tests that actually test what you care about and Swing the focus towards writing tests so that when you refactor code You can actually confirm that the code still works If you only have tests that don't do that you're missing a big chunk of your test suite I personally call them regression tests. I think that's the right term Another thing that people say is I have a crazy example. I can't it just doesn't work that way, right? Someone I have a spec that's 300 pages and I implemented in a model in my app and you're telling me I have to you know push through my app 300,000 Examples just to test all these cases and the answer is no you actually don't have to do that Right, you should treat it like a library or a web service You should write a bunch of tests that test this piece of functionality as though it was a library that we're using And then you should lock down the API and only use the exposed API so Don't treat it like another part of your app that is sort of fungible and subject to change treat it like something that you give do thought and Care about the API lock it down Commit yourself to the API. So if you refactor in the future, don't change the API that's being used in the rest of your app Treat it like a library. I would I personally usually move those things out into libraries, but maybe that doesn't make sense in your case bottom line is Yes, sometimes it doesn't work that way, but in that case you need to be damn sure that your interface is solid and locked down bottom line When you modify your app you need proof that it still works, right? That's the one thing that I want people to leave here with You want to have tests that confirm that modifications to your app Do not break your app If you don't have tests that do that and I think that's where Hampton is coming from He doesn't have Hampton doesn't think tests are very useful because unit tests don't do that And he wrote a ton of unit tests and found that he broke his app and nothing failed, right? So you need You should be thinking about what proof you need To prove that your app works and what proof what your app does not work Your regression suite should be robust my personal rule of thumb is that I don't count unit tests towards coverage Now I don't mean coverage the metric although. It's a nice metric I mean that you should care that your test your app is covered right that your app is tested And I don't count unit tests towards the coverage. So if there's a bunch of weird. There's a bunch of Conditionals inside of a method. I actually test all those conditionals through the full stack testing Sometimes it turns out that I've written my code really poorly And I should probably not have a bunch of conditionals that are 50 lines long in a method and it turns out that when you Break it up. It's easier to test, but we already knew that Okay, so that's the end of the Talk on that. I'll take questions in a minute. I have some useful tidbits and some announcements The first one is Ruby gems DLL help so we're actually uses a lot of gems Why we want to be modular we want to care about dependencies and it turns out that Ruby gems has some problems And I just want to explain what those are so people know and can like spread the word talk about potential solutions So here's a problem imagine you have a A gem called foo that says gem baths you have another gem called bar that says gem baths equals 1.0 And you load them in that order Right what happens is that foo when you say gem baths it loads baths 2.0 And then when you load bar fail right because it needs 1.0 But what's what's the problem here right there's actually a way to resolve both of those dependencies Such that it actually works right because foo doesn't really care about what version it is So a foo loaded bar 1.0 everything would be happy and copacetic the problem is that we load our gems in a list in order Right so and this happens a lot to us because we have a lot of gems and a lot of and we use gems as our dependency system I think this probably happens in rails. You just don't get a warning Right so plug-ins could have dependencies on each other's versions and they could break and you just don't get warned when they break just not I don't think is a better solution so What is my suggestion my suggestion is that Ruby gems has a facility in it should have a facility in it Please 1.4 that lets you give it a list of gems and then it will resolve what dependency So instead of just loading a list of gems in order, which will break which breaks the way I showed Allow us to give you a list of all the gems that are going to be required and it will resolve what is required We murb already has a system called dependency which will which defers the loading so we can easily make use of this other people Can make use of it simply by Instead of having a bunch of gem lines having one gem line basically it's what I'm suggesting But it's actually a real problem. That's only going to get worse if we don't address it head-on We've had a lot of problems with and Ruby gems is not very helpful in terms of warnings and error messages Sometimes it'll just say can't load bar when it really means bar required a different version of Baz sometimes it'll say can't activate bar and Won't tell can't activate Baz 1.0, but it will tell you it won't tell you where it came from right? So in addition to its sucking it's really hard to track down where the problem comes from But the proposed solution is just give us a facility for resolving a lot a bunch of gems at one time That would be very helpful The other thing is don't use Ruby gems 1.2. It's really bad bunch of bugs It exacerbates the DLL hell problem and you should upgrade unfortunately has another bug That prevents you from upgrading it through gem update minus minus system So the worst version of Ruby gems ever can't be upgraded You have to download it what the reason I'm saying it to this room is that I want you to go out and tell all your Ruby friends to upgrade to 1.3 immediately the bug that it has is that it treats development dependencies as runtime dependencies But doesn't install them so for instance a lot of gems use hoe hoe insinuates itself as a development dependency So let's say I use parse tree parse tree has hose development dependency In Ruby gems 1.3 that just means if I don't actually say minus minus development when I install it It doesn't get installed and it doesn't get nothing happens at runtime in Ruby gems 1.2 What happens is that it gets treated at runtime as a runtime dependency So if I try to require parse tree at runtime, and I don't have hoe installed it Explodes even though when I installed it it never bothered to install hoe in the first place and gives a really impossible to understand error so That happens a lot. That's one example that we've seen a lot But there's a whole bunch of other examples that for which it could happen as people could start to use the development dependencies more So just don't use 1.2 Please download install and get other people to install 1.3 some announcements Murb 1.0 final just being released today so we went through last night the lighthouse and cleaned up any bugs that were deemed showstoppers and Deferred things that weren't and we're releasing today The second thing is engineer. It's going to be offering Murb support that means support from me. It's pretty awesome. I think So you can go to engineer.com slash Murb support and sign up I think we're starting the offering on October 17th because I'm taking a vacation until October 17th or November 17 But you can go there and sign up and We'll get we'll give you more details the long and short of it Is that there is going to be basically priority bug bug reporting so that you can report a bug and get Quick turn around answer about what it's wrong what a workaround is and perhaps when it will be fixed And then a per-incident support as well Which gives you the ability to actually open an incident and talk to me and other murb core team members About what exactly is going on so and you know it's gonna offer Murb support and can go on Can I do we know? We're still working it out The second thing is that MA Agile, which is run by Matt I'm in Eddie MA who is a Murb core team member is going to be offering Murb training in quarter one oh nine I Yeah, you can go check it out at MA Agile comm slash training sign up. It's roughly patterned after the Agile studio with a pragmatic studio Approach and you can go check it out and it's gonna be cool and I think we're MA Agiles also Be happy to set up if you have a corporate need for training. Basically, they're getting into the training business. So I will be doing Training for in quarter one and then you can hire them to do training as well So that's pretty much all the announcements and then I will a few minutes I'll take any questions about first part of my talk or the second part. Yes. Ah, yes How did I forget that? Yeah, Murb Day Atlanta, right is going to be December 6th. I'm speaking there and a bunch of other cool people are speaking there and you can find out at murbday.com, right? Atlanta murbday.com cool Any other questions? Yes, sir So one of the things you're talking about is trying to cover all of your cases with these basically functional tasks And if you've got four sources of freedom each source of freedom has four different outcomes to cover a method that uses Excuse me that uses that you've got 256 possible tests. You need to test How do you deal with that computational explosion of cases if you want to cover everything from a functional event? Well, so so that would be true if you were trying to test if you were doing Full functional tests and you wanted to test every possible scenario But you don't do that in unit testing anyway, right in unit testing you can't get the computational explosion So let's hold on so let's assume that we don't care about the computational explosion We don't care about testing all 256 cases if we don't then we could for each Degree of freedom we could use one Only one of the possible scenarios in the other ones. So for instance So so well, I think it is useful to cover as many cases as possible that isn't actually the point of what I'm saying What I'm saying is that you should test you should test each part of your code through the public interface So for example We test render render has takes a bunch of options render could interact with content type It could intense it could interact with a whole bunch of other With provides right and it's probably a million different options But when we test render all we care about is that the render method works the right way So for the purposes of render we test it as though it was you as though we were unit testing it So as though we were testing the method, but we don't test it through Calling render we test it through a real app And Yes, that's not as ideal as if you could test a million cases, but that's out of the question anyway We're not gonna be able to test a million cases. So We think that it's superior to test Your kit the cases where it would be very difficult to test the common soil explosion through the stack Right through the black box instead of by testing them Individually because then they could actually be used as regression tests We can actually confirm if things break and we can refactor A couple of those cases through the larger stack you would not so what we would do is we would we would do units Okay, let me step back. So what you're basically what you're asking is if you do real functional testing where you try to handle all scenarios Right, there's too many cases. So you want to be able to test just render Just a render method. That's what you're saying, right Sure Right, so you either you're testing all cases in which case just do that through functional testing Or you can't do what I think you're like arguing a speed thing. Is that I think there may have been conflated arguments I'm sorry Are there any other questions? Yes That's exactly right. So what I said a few slides back was that we actually don't I don't think and I don't think most of us think that unit testing is bad Right, we actually find it to be very useful from a design perspective. I think I said that It's just that you We think that it's perfectly fine to delete those and you have version people say oh, I want to keep it around for documentation I hope you're using version control and if you are you could just go back to when you made the method and go read the tests Right. Hopefully you're also documenting your code. Well, but yeah, I that's definitely a common pattern where you And Well, it didn't change the API Right, we put the point of the o9 rewrite was not to change the API but to start but to basically rebuild it from scratch Architecturally so I would have hoped that the o9 test could be used in such a rewrite The o5 test could not Yes Only heard like three quarters of what you asked So I don't know right a test that you don't know what it should be doing Yes, I think you're asking how to do a certain kind of test which is you want to confirm that your your code complies with the spec Right is that basically you basically have to write you have to go through the spec and write a test suite that Is a test suite for the spec and then you can use that Ideally you're using a library that does something like that. So If you're if you're if you're talking to an API or dealing with a spec Hopefully you're using an XML parser that's already XML compliant and you could trust their specs Yes I Can someone pay for more support and get help with JRuby? Yes Merb Merb is a hundred so okay. I should actually address the JRuby thing in general We spent probably 20 to 30 percent of our time over the past two or three weeks in the run up to 1.0 Dealing specifically with JRuby issues. We're very committed to making Merb run on JRuby. There's one current Problem which is that we the database driver that data mapper uses which is what we use doesn't run on JRuby but I think that gap is going to be closed quickly and then it everything should work and Yes, we're very committed to it running on JRuby and any other Ruby implementation that runs rails my criteria and Basically we support I would assume would cover that as well other questions We should have it's too late now But on that topic we are we will be running the 1.0 the specs that will be released with today's release against every Future version that is in the 1.x line So we actually believe that our specs do what we say they do which is serve as a set of regression tests It's entirely possible that we will have failed and there will be some non regression tests in there But I think we'll be very loud and announce clearly that we have to change the swag suite The idea is that the spec suite for 1.0 actually confirms the API of 1.0. We're not going to change it till 2.0 Yes Very useful for designing someone else What's my take on test-first development? I think I personally find test-first development to be very useful I think it's sort of orthogonal issue to regression suites. I don't think test-first development produces very useful regression suites I think test-first development is very helpful for figuring out how to design something and that in the same way that you Frequently write one and throw to throw it away in real code I find that I want when I'm done with my unit tests which helped me design I want to write a set of regression suites that actually tests in the end what I care about I don't want a bunch of tests lying around that test things that I ended up not caring about in the end