 Thanks everybody for coming. My name is Brian Ford and I am very fortunate to be one of the Rubinius developers working with Engine Yard. I want to give a big thanks to Engine Yard for the support of Rubinius and also for RubySpec. We started RubySpec back when Rubinius was just an open source project that people contributed to in December of 2006. And I worked on that for a little over a year. But Engine Yard has been really great in helping us. And RubiSpec is something that benefits everybody in the Ruby community. My talk is what does my Ruby do? And the reason I named it that, what does my Ruby do? Is because everybody probably has a preference for which Ruby that they are running. And they put me right in the middle of the implementer's track but this isn't really about Ruby implementation although it's very important to it. It's also relevant I think for everybody who uses Ruby. Because as you write your Ruby code and you want to run your Ruby code, you want to know if your Ruby code will run correctly or at least the way you expected it to. So that's a pretty big number. That's probably more times than I've heard Maverick and Socialist in the last six weeks. But neither label being accurate of course. That number is right now how many things that we call expectations there are in RubySpec. And we'll get into what that means but just on the face of it, that's a big number. And when you think about your Ruby code and the number of different combinations and the ways that you could put together those 30,000 things, it's pretty huge. Before we get too far in though, I have to give this disclaimer. This project more than any project that I've ever worked on has been a bike shed project. And it's convinced me that there's no way that you will satisfy everyone, which is a good thing to learn early in life. So the things that I'm going to explain are the rationale for some of the decisions that I've made. And some of the background that you don't see if you just go check out the project or look at the documentation that we have available. But recognize that it's my opinion about the way things should go based on my experience primarily working with Rubinius in terms of implementation. But it is a big bike shed. Okay, so RubySpec, what is that? It's an attempt to make a complete executable. This really sucks. This doesn't work from download specification for the Ruby programming language. And we're going to look at each one of those components, except you guys already know what Ruby is. The current status, there's this number of things. Files are what you would think of as files when I explain what they mean. 9,203 examples and 30,866 expectations. An expectation is every time you say should or should receive, it's an expectation that some bit of code behaves some way. And examples are groups of things. I know that when RSpec was just starting out, there was this big talk about one sort of expectation per example, which was a really important idea in that you were testing a facet of behavior with each thing. But Ruby is very complex and sometimes it makes more sense to have a little coarser granularity. So instead of having 30,866 examples, which we certainly could, that's been made a little bit more manageable by looking at those things in collections that we call examples. The goals of the project are basically these four. One definitely is verification in that if you say that you support Ruby version 1.8.6, how do you know that? You run this and if you fail somewhere, there's a problem with the spec or there's a problem with your implementation. The other one is compatibility. It just comes right out of verification. If I want to run my code on JRuby on Windows and it runs fine on Rubinius on Linux, this is an avenue for verifying that. Development, that's the stuff that probably everybody that's been in this room is really interested in today. Implementations, it's an effort to drive the implementation by identifying what you need to do and then do it instead of writing some stuff and trying to figure out if it actually worked. And finally experimentation, there's that old saying that there's as many LISPs or LISP implementations as there are CS students or something like that, like everybody writes a LISP. That would be a dream come true if every CS student was writing a little bit of a Ruby implementation. And experimenting with ways of making Ruby faster, that would benefit everybody tremendously. When I visualize Ruby spec, this is kind of a humorous picture that comes to mind. It's a safety net underneath you and it could be under your program or under you, sitting there typing in your chair as an implementer. So the structure of it, we use hierarchies obviously to manage complexity and that's what we've done here in that we have the very top is Ruby spec. Underneath that we have broken it into provisionally 1.8 and 1.9. I did this early on because I had no clue really what 1.9 was. It was a nebulous cloud over here and I knew that we were going to target 1.8. This was December of 2006 and Rubinius could do things like add two numbers. So I threw stuff in 1.8 in sort of a sense and this actually came a little bit later when we broke Ruby spec out but conceptually I put things in 1.8 because that's what we were targeting. I would really like that to actually go away and see something like this. So the three divisions underneath there are core language and library. Core means everything like array, hash, fix them, big num. Language is all the keywords essentially in the Ruby language. Things like while and if and the library is everything that you have to say you require law and it comes with Ruby. The files that are in those directories look something like this. Index underscore spec if that was in the core array directory then what you're talking about is the index method on the array class and that correspondence is primarily to give you a very easy way to index and say I'm looking at this bit of Ruby code. I want to know what this bit does. Is it a method? Yes. What class is it on? Go there and you find your method. We try to make it very consistent and there's a script that we actually have that will spit out including turning symbols like array reference into element reference spec. I'm going to go back for just a second. In the language this was a decision I made but basically when I think about Ruby I think about things before I think about actions because it's an object oriented language and we make objects and then we send messages to them essentially. So to me objects are a bit more primary than functional things. There's many ways that you could slice up the pie that's Ruby grammar based on assigning to a variable or any other sort of action that you think you could take. But instead of trying to make my idea of the behavior of Ruby match someone else's we simply have things like keywords. Therefore while underscore spec has specs that deal with while and if and it's pretty consistent. There's a few things that are clumped together and doesn't make any sense all by itself. So it's actually sort of subsumed into the if spec. By the way you're welcome to ask questions. I have like three different talks jammed into one. I will try to go through these and have time for questions but if something is burning up your curiosity please ask me. So I didn't dream up the idea of testing Ruby. There were people that came before me and when I looked at what they had worked on which was good work I kind of saw something like this and I did not think that I could polish that stone wheel in a sense into something that would be really good for what I was trying to do. I really wanted a wheel that would go on that car which is an awesome car. So credit goes you know especially to Ruby itself has tests. The first time I built Ruby you know I saw the test directory make tests zip okay it was done I was wow everything works. I didn't know what the coverage of those tests actually were which is an issue but Ruby had tests there. So and we've ported a lot of those to Ruby spec. Ruby test was a project that's also very interesting and BFTS I don't know exactly what that stands for but it was a project to make a better Ruby test I believe. So credit to those those projects. Why are spec like why not just make BFTS or Ruby test better and it comes down to this reason right here. Why are spec like. If we look at a typical R spec assertion there's three parts. There is an element to that that groups. There is an element that provides some sort of description and then there's a code example. So describe is a way to set up a context of sort of a universe of discourse in a sense. There's the it which allows you to associate textual almost like metadata and then the code example. The R spec syntax is I think a crucial element of it because when I look at assert a comma B in a beautiful language like Ruby it makes me you know cringe. No one can agree and even though everybody can say that expected comes before actual there's no logical sense that it necessarily comes before and there's other unit tests that flip those around and every time I do something I can't remember exactly which way it goes. I just I hate that syntax in Ruby something like a dot should equal B couldn't be in my opinion clear. So that's what I like. And oops. The debate over whether it should be a dot should equal B or a dot should dot equal B I think is moot R spec one. I don't really have a strong preference either way but I chose the way that R spec uses for the legions of people who are writing R spec specs for their code and I want them to contribute. I want them to understand it and so bacon is a great framework I don't agree with you know taking a very small piece of the pie and saying that's what we're going to do. So formal specification we had some great news recently colleague of mine actually informed me of it because I don't read Ruby Ruby talk that much but there's an effort to write a formal specification for Ruby which is great. And I haven't really had a chance to talk to mods about this yet but hopefully will. I mean it's a really exciting thing I think. Some people have you know an opinion to the contrary but I think it's a good thing overall. A formal specification is going to look something like this. It's going to tell you this thing should do something in a sense because what it's basically doing is distinguishing everything that should be alike from everything that should be different. I mean it's just a matter of basically drawing lines in the sand and say that if your lines look like this your Ruby they don't, you're not. It's probably going to be a little bit more formal as soon as you add some numbers then you have you know something that makes more sense I guess. But it's basically going to be the same sort of thing. What you might realize if you're ever done dash fs while running mspec or RubySpec is that it looks very much like what you could actually generate from the spec suite itself. And so I think the idea that you're going to write a formal specification without examples of code is never going to work because somebody is going to be implemented, they're going to go read the spec and they're going to go write some example code and that's exactly what that is. So I think those two go together and I think that I'm hopeful that the body of example code that we have right now will become part of a formal specification. I don't know what barriers there are to such a thing but that's sort of my rationale for why. So the really challenging part of writing the RubySpecs is dealing with things like there's different platforms, there's different implementations, there are different versions, platforms have different operating systems on the same hardware, they have different operating systems on different hardware, there's BigIndian and LittleIndian and so many different ways that you can put those collections together. And you can take the perspective that that's metadata to the examples, like if A.should equal B, A.should equal B anywhere that you run Ruby code, otherwise what are you running, right? Unfortunately it doesn't work that way. As soon as you get into file handling or BigIndian versus LittleIndian issues when you have pack and it's got a specification for it, you're going to output that LittleIndian, there's a lot more to it. So instead of being metadata, I decided that the guards need to be a part of the spec and that the spec explains exactly what will happen on a particular platform. So we have a guard, platform is, for instance, windows, and inside that, that code will execute if that guard recognizes that what you're running on is windows. We've got the inverse of that because we're using blocks and blocks are not simply conditionals. We're using blocks like a very simple if. We're using them because they're sort of logically consistent with the describe and it. And so instead of introducing, like say, you could do that with this real Ruby code, but it looks, when I looked at different ways of doing that, and for instance, the Ruby test project has guards that use things like if, version, something. When I looked at that, I didn't like the disconnect. I like the more uniformity with the blocks, and so that's what we're using. The platform is guard itself. If we go look at the code, also hides a lot of complexity that I think will tend to bleed out. I mean, you can abstract that into a method, a class method, whatever else, but I did it mostly for the consistency. One of the things that really complicates implementing the platform is guard is identifying what implementation, and there's a couple other guards that's really important. And so we've made this strong push for this variable, this constant Ruby engine. And you can describe it as all different sort of things, but to me, your implementation is what makes your Ruby code run. That's kind of like an engine that makes a car run. So for whatever reason, whether it's clear or not, we've basically said we really need this constant Ruby engine. But when we've discussed it, it's not always understood what it means, and basically what it is is simply a constant that the implementation sets. If it tells you that you're JRuby, then what you can expect is that you're going to behave in the spec like anywhere where you state in a guard that you're JRuby. So it just takes the ambiguity out. And if you go look at the MSpec code, you'll see that there's hoops that I jumped through. JRuby runs on Java, and I'm sure Charles can explain this better, but basically when you're running on top of that, in a sense, your platform is Java, not Windows or Mac OS X. And so we jumped through some special hoops in that case to say, yeah, we're still on JRuby. So this is the reason for Ruby Engine, and I'm not sure what Iron Ruby would actually call it. These are ones, of course, Ruby VM slash 1.8, it would just be Ruby, because that's what it is. The version is something that I tried seven different ways from Sunday to keep out of the spec, but Charles convinced me, and I think he's correct, that it has to be there, and it has to be persistent. I wanted version stuff to fall away as we marched forward. So if today array at returns a fixed num and tomorrow it returns a string, we fix the spec to expect a string and we forget everything else. But Charles explained that you really can't do that when you have to support more than one version, and when you have an orderly end-of-life process, you need something like this. So the guard as Ruby version is, it takes a string that it parses into a number so that it can do things like that. So you can use the inclusive end or exclusive end range, and you can say, in this particular case, it raises a local jump air, it's going to be true from the beginning of time to 1.8.7. Another guard that is really important is something called Ruby bug guard. This only executes when it identifies that you're running on what we call standard Ruby. So this introduces the idea that there's one standard for Ruby, and this guard will not execute what's in that block if it identifies that it's Ruby and it matches the version, which is a second argument to it. The first argument is a string, and what we'd like you to do is put the red mine tracker number for that bug or the earlier Ruby core, Ruby talk, however it was sort of tracked, a number that identifies that so that we have that correspondence between there. And there's a bunch of guards because there's a lot of different ways that these things can look different. I mentioned the endianness. There's an extended on-guard for implementations to be able to put in there what they actually do differently. There's something called quarantine, which says don't absolutely ever run this thing because something really bad is happening and gives us a chance to say, okay, now let's investigate what happened instead of people going, oh my god, it's segfaulting this, we have to tear it out. Instead of like, what I try to do is keep it very conservative when we mess with specs themselves. And so what these guards do is sort of twofold. One is they make it explicit what something is expected to do, and then they make it more formal also. So there's a bunch of them in there. Okay, if you are a CS student, which I was and I'm no longer, or if you've studied programming, you should have learned this math. If you graduated from high school, you probably learned this math. So whenever people start talking about math, we often get glazed stairs. Here's some math, and I think it's really important to just sort of cover briefly. I'm not trying to be pedantic, but I want people to think about this. We define a function f from a domain A to an image or range B. And we say that that function is on to if the image of f is equal to B, whatever we're calling that, and they don't agree whether it's range or co-domain or whatever, but the idea is that anything in A maps to an element in B. Then we can look at our domain, which is if we think about we're testing a Ruby method, all the possible inputs. So let's say array at, it can take things that look like numbers. And if we think about everything we could possibly send in there, and if we can break that into a set of distinct sets so that you get everything back when you put them all together, but when you take any two of them, they share nothing in common. So what you have then is what's called a disjoint partition of A. You can typically take that one step further if you can define what's called an equivalence relation, which you can because any partition is an equivalence relation. Then you can only take one thing out of every one of those disjoint sets, and you can test that one thing. And if you've done things correctly, that one thing will assure you that every other thing in that set when you give it to that method is going to give you something in the range of that particular set. And so this is a little bit of a side thing, but when we look at RubySpec, one of the things that we need is more people looking at those specs, understanding how the method behaves, its various facets of behavior, and ensuring that we have sort of done this process correctly. So it's a huge place that we need a lot of help and people's attention. If this doesn't make sense, I mean ask me about it later, or just do a little bit of research, but typically basic math through algebra is going to teach these concepts. So apologies if that was boring, it's helped anybody. Did I say complete? Yes, the goal is a complete spec. How complete are we? I have no idea. I can say that with Rubinius failing 500 or 600 out of those 9,000 or whatever examples, let's say, we run a lot of Ruby code. So we must be doing a lot of things correctly and we're passing a lot of specs so we can kind of assume that there's some correspondence there. But one of the things that we need to develop is some way to understand completeness. You look at typical coverage, you can't just do code coverage and say, are we complete because we covered all this code because which implementation do you use? We can say we use MRI 1.8, but then you start questioning, well, is there overlapping facets of behavior that as soon as you implement them in a different way, will you no longer sort of have that overlap? So it's a challenging question. And oops, what about accurate? So we write all these thousands of things, I mean, they run, that's some assurance that we've done what 1.8 says it does, but is it accurate? This is where we need a formal audit process for the specs. And it's open for development. So I'm going to move on to the next, but before I do that, I have no idea where my talk is supposed to end, so let me just check that real quick. 255, okay. Okay, we're changing gears now for the RubySpec project. I'll tell you a little story. When, you know, from like December 2006 until I would say like April of 2007, there's a group of us in Portland, a small group that would kind of get together and we would, on the weekends, and we'd try to do something with Rubinius. And of course, wanting it to run the specs, I wanted it to run our spec. And we labored and labored and labored. And it's not that our spec is bad software. It's just that our spec uses complex Ruby features. And you don't know how hard it is to implement some of those. I mean, one time my laptop, it like used up all my disk space that was free, essentially, because I was trying to trace what it was doing to figure out why the hell the thing just went off into space when you try to, you know, load our spec. And I could not, for, you know, hours upon hours, narrow that down to a reproducible test case that I could understand. So what I started writing, after talking with Evan one day, and I said, hey, how hard is it to implement, say, passing an argument to a block? And so we sort of looked at this very minimal set of features that you need to support something like the syntax that our spec provides. So I wrote this thing that was kind of a mini-our spec. And I started calling it mini-our spec because I had no intention that it would ever take over any of the duties of our spec. It was merely a crutch to get it somewhere. And then Ryan Davis started giving me a lot of shit. What's the M in M-spec? This isn't mini in my book sort of thing. And so I decided that that's the M in M-spec. I hope you guys understand what that symbol is. So M-spec is actually a very purpose-built little bit of software to enable us to run the specs on Ruby implementations that may only be able to support a few features like methods, classes, and maybe yielding to a block. It's programmed really simply. I don't use inheritance anywhere in there because included modules, which is something that's essential to doing inheritance, right, were really hard to do correctly in Rubinius. And partially, I was going to say, I didn't really understand. It was hard to debug, but I understand a lot better. I still know that they're hard to do. They move away from that. It's also sort of event-driven in a sense, and it's pretty modular. That said, there's a lot of files now. When I rewrote it like the third time, John Lam said, what the hell is this thing? It's so complicated I can't figure out where anything goes, that sort of thing. So there's a lot more files, but each piece of it, if we look at it individually, is still very simple. So still kind of begs the question of why not R-spec? Why not just identify a better set of R-spec? Why not contribute back to R-spec? These are questions that I've had, and some smart aleck basically came up with this sort of quote. And that really summarizes the idea that R-spec is being used in applications all over the place doing all sorts of complex stuff that are way more complicated than RubySpec, even with 30,000 whatever things. I don't worry about namespace clashes. I name something spec context. I don't care if you have a spec context in your application, because Ruby does not have any spec context in Ruby. And those are the sort of decisions that make it possible for me to do things in M-spec that you just can't disregard so easily in R-spec. So it's a very different problem domain, and that's why. And I think Dave and everybody who's contributed to R-spec, I think they're doing wonderful work. So I applaud them tremendously. I've never contributed anything to R-spec because I don't think that it's a fit for this particular problem. That said, we're trying to make every effort to have the specs execute correctly on R-spec, and I haven't tried that recently, but that's definitely a goal. I'd like people to be able to just say, you know, spec and do something and have it work. Specs are kind of complicated. You can do something very simple, where you just yield to the block, and you check that, say, it raised an exception or not. What I implemented instead was something that has a whole bunch of events that sort of happen. These are all the things from before any specs are run through loading a file, because an exception can occur when you load that file. So you have a syntax error, and those were just sort of falling right through. Every one of these events, I added only after it became necessary to like sort of either tally something or give the user some sort of information. And so these are the sort of, as you go through executing the block that is described and inside that executing the blocks to the it blocks, these are the things that I found to be important events along the way. M-spec itself is made up of, as it's executing, these sort of three ideas. There's filters that say yes or no to execute a spec, and if you've used, say, dash e in R-spec, you know that you can say execute a spec that matches this description. There's actions that are things that can occur. It notices that you loaded a file and sort of do something, and there's four matters. That's where it collects up all the information and either outputs it to you while it's running or at the end. So here's a very simple example of a filter. It's going to check that the description, you know, when you put the describe plus the it together, every time it goes to execute one of those blocks, it's going to say, does it match this? And it does a ridge x match on that. Escaped, ridge x. So if it's got special characters, it's going to get escaped. We have another switch that will allow you to pass in raw, ridge x. We have another thing that will match, say, tags, which I'll talk about in a minute. But that's the idea of a filter. All those things are implemented as a filter, and that's just something that says, yo, I want to be notified before you run any spec. And if there's a filter in there and it returns false, the spec doesn't execute. That's all there is to it. Spec debug. This is an example of an action. Our debugger isn't working right now in the new C++ VM, but it will. And the idea here is that before you hit that spec, drop me into the debugger so I can step through the whole thing. And it's found that being really useful instead of going in and putting in prints or trying to break into execution. It just basically executes our debugger right before it goes to execute that spec, and you can step through it. And finally, the formatter. We have the spec doc or spec docs, whatever. Output, which is kind of cool, gives you all the text. We have the dotted formatter. It's down in Australia. And at its brand, it was really cool, a bunch of brand new guys that were contributing. They got tired of all the dots, and so they made this, which is just kind of fun. It actually uses ASCII color codes, so the F-turns red, the E-turns red, that sort of thing. Tags for all those regex fiends in the audience, and for the rest of us. That regex matches this, and this little thing has a few parts. Fails is a tag, and what you can do with fails is say, do or do not run a spec if it's marked as fails. It can have a comment which go in parentheses. The colon is the single delimiter. That breaks that apart from the rest, and the rest is a full description. So take the described text, put the it text onto it, string that all together, that matches. So fails, if it says don't run fails, it will insert a filter. It will have that full text. If that full text matches verbatim, that doesn't run or does run depending on what you ask for. To apply those tags to specs, we have a tag command, and what it creates is a file that shadows the spec file in a different directory. This is metadata for the specs. It doesn't have anything to say about the correctness of the spec or anything else, but it's metadata in our implementation. Say we fail that. That doesn't mean anybody else fails that. And so there was a need for a way to identify those things external to the specs, because we don't want to go in and muck with those. Other people depend on them. Last resort, you go in and muck with those specs. But we need a way to not say segfault when we're trying to run a group of specs. We can omit the ones that are, say, failing or unstable or incomplete, which brings me to another big bike shed sort of issue, and that is why not support something like pending. And MSpec does support pending in a way, but again, some smart alec, this is his opinion of pending. It's an incomplete solution. But there's always a but. It's often good enough, and that's, you know, it's true, because everybody tells me all their reasons why pending is a great idea, and I hear them and I understand what they're saying, but I don't agree that it's valid for RubySpec. And the reason is, is because, you know, I don't care what you do with your private clone, but when we push stuff to RubySpec, it either is a spec or it's not. And a bigger reason is that there's real need for spec metadata. I'll get you in one sec. But it's metadata, and whether something is pending or not does not say anything about the accuracy of the statement that you want to put in the spec. It's metadata, and we have tags for metadata, and it's a much more robust solution that has a lot of different uses, and we can grow it. Pending, you can grow. So, yes. Does this have anything, or rather, if somebody is new to this and wants to write, would like to contribute specs? Does this reflect on the front door for people to start where we, where to start to get writing spec? So, are you saying if we had pending in the specs? Okay. So, the question is, you know, in particular, if we have pending, would that be a good marker for people to sort of identify a location to contribute or, in general, what's a good way to contribute? There are specs. I will answer the first one categorically. No, I don't think pending is a good place to begin contributing. We do have the ability to generate stubs for every method on any class that you want to hand Ruby, and we typically run that in MRI, in Montza's Ruby, and generate those stubs with a comment that this spec needs to be reviewed for completeness. And the reason I say that that's a better solution is because if you've written one spec for one method that may identify one facet of its behavior, there's not a way to be able to say, I think I'm done with this. But if we leave a marker there, and you could say pending, but the reason, again, that I don't agree with pending is because we can attach tag metadata to any spec, and we can attach it just as easily to a spec like this needs to be reviewed for completeness in a variety of different ways without overloading the word pending. Pending is one word. We have an infinite vocabulary in our specs, and that's why I disagree with using pending. So in general, what's the best way to contribute? If you know anything about Ruby, pick some part of Ruby, either some part in your application or some part that you're interested in, does it make sense? Does it make sense relative to what the pickaxe or RI says? Does some bit of your code not work right in a particular implementation, and you're trying to figure out why you go right this way? I mean, it's basically very wide open for how you would actually begin to contribute. Run the specs on whatever Ruby is installed on your program. There's a freaking million different on your OS. There's a bunch of different versions of Ruby. 187 and stable 186 are running a lot cleaner. There's still things that we haven't identified in terms of version differences from those older ones, and we'd like that to be very accurate. My understanding is, and we can ask for an official word, but 1.8.5 is no longer maintained. I don't want to add any Ruby version is 1.8.5. 1.8.6 in many different varieties, and of course obviously stable is out there. All those are areas that you can look at, and that's it, yes. What about pending about the locks? So it goes straight. Yeah, you can leave those in. MSpec ignores them. If you put an empty block, MSpec will raise an exception that there's no expectation, and that actually helped us find a number of things where someone had done A equal equal equal B instead of A.should. So pending, if you want to leave pending with a description, MSpec ignores it. And that's as far as I will go unless someone holds me down and whatever, it gives me something into the pending territory. So real quick, RubySpec is really just beginning. You know, if we're 99% complete, we're really just beginning because there's some really neat ideas, and these might be like far out ideas or whatever, but these are some possibilities that I think are realistic and of great interest to everybody who's working on implementing Ruby. You know, what if we understood, in a sense, a little bit more about what Ruby code means? What if you could do something that analyzed your code and identified specs that they think that this program thinks you should run? Or what if you say the something should, not should, but the something does this or that? It analyzes that list of specs and identifies code, that's kind of weird, you know, code that your program would actually run. Or analyzing the standard library. Ruby, a lot of talk about, you know, and Ruby about duct typing, and it's very useful, but is the use of say, two string in the core library consistent in that can I pass anywhere that you expect to act like a string, something that implements two string and expect the Ruby core library to handle that correctly? And a lot of these things are like higher level views of how Ruby behaves that we don't really have right now. A lot of like little example code, it doesn't work this way, why doesn't it go read the code, go do something else? I think that there's a lot to be said, source code, transformation, and understanding more what is going on in there. Finally there's, you know, the idea of collecting performance information while it's running, the overhead of MSpec for instance is going to be consistent. Statistically you could end up extracting that information out particularly what type of code is being run and how long it's taking. We want to add an action, a benchmark so that you can keep a private benchmark of the Ruby spec, and if you are then implementing something in the VM and you run that and suddenly, you know, you've like 10xed execution somewhere, it's going to be really apparent that's relative to your private, you know, sort of benchmark of what running the specs are, but it helps us identify further vectors pass or fail in a spec. You can find out more about RubySpec at rubyspec.org and it is a GitHub project, RubySpec. I think it would be really cool someday, it's not complete yet, but I would love for you to be able to do gem install mspec, gem install rubyspec, mspec, and it runs all those specs. I don't think it's really realistic right now because it's not very complete, you'd have to be doing gem update a lot, so get clone, do mspec on your RubySpec clone. Please contribute. We've had some great contributions, Google Summer of Code, I want to thank a few people, Arthur Schreiber, no karma on IRC, Federico, I can't pronounce his last name, F, and it's B-U-I-L-E-S on IRC. Two Google Summer of Code folks who did a great, great job. Yulii-san is the 1.9 maintainer and she's been helping us get 1.9 specs in there. Awesome, because there's a dearth of those specs. Charles and the JRuby guys, Iron Ruby Jim DeVille and Iron Ruby has been awesome, he's helping us understand how things work on Windows, I have no idea. So if I missed anybody, a big round of thanks to everybody who's helping out my colleagues as well. No contribution, it's too small, but a little bit of a rant. Please don't send me pull requests. I think that get hubs make a clone easy where I can push to is great. I think that the fork thing is way over the top. What I end up seeing is repositories where people were figuring out how to use get and I can't make heads or tails of what they have. If you can decide this is what you'd like to see, please use Format Patch, put a ticket on RubySpec, it helps us track things better and I curse less. So, thank you very much and questions. I have a few minutes. Yes, Wilson. You can run a lot of the specs with just yielding to a block and I once made a very, very simple thing that just described yield and, you know, I had a few minutes and I had a few minutes and I just described yield and, you know, an expectation that says actual, equal, equal, expected, raise an exception if it doesn't. If you can implement a proc where you can, you know, objectify a block and save that away, then you can collect a whole list of things and then run them, then you get before and after actions, which is great. So that's a step up. In order to be absolutely correct you need InstancyVal. I should have that particular spec and constant lookup. It does not work if you call the block. You have to InstancyVal it. So at some point you have to support InstancyVal. I tried for the longest time. Not ever to have to require that, but sometime you will. And if you're working on implementation seriously, if you're like a, you know, grad student or somebody and you're like, I want to work on a Ruby implementation and you want to run the specs, talk to me, we can write a very simple spec that will allow you to run tons of specs, a very simple like MSpec type thing that will allow you to run tons of specs until you can bootstrap up more. Anyone else? Alright, thank you very much folks. I think I'm out of time.