 I'm going to go ahead and get started. Hi, my name's Eric West. Before I start, I just have to say thanks. I had a tragedy this morning. Discovered it looked like the insulation had been burned through where the wires had gotten frayed. Pretty horrible. But thankfully, I got a little help from some friends and just wanted to say thank you. My name's Eric West. I'm a Ruby developer at Lonely Planet. Former JRuby Google Summer Code participant and currently the maintainer of Arsense. I've got a few goals for this talk today. I want to explain what Arsense is, encourage you to get involved, and kind of a bigger goal of motivating you to stretch yourselves as programmers. So what is Arsense? Arsense is a type inference tool for Ruby. It reads in the source code and then it provides information about that source code. And that means we can take that information and build tools to do really cool things like code autocompletion, find by definition, error detection, especially type-based error detection, and automatic refactoring. These are all tools that are typically available in statically typed languages and Arsense makes it possible to bring that kind of tooling to Ruby. So where did this start? I started programming about three and a half years ago now and I doubled a little in Java, did a little node, and then came to Ruby where I fell in love, found home. And the Java showed me, I guess, what was possible. So when I came back to Ruby and I started using an editor called Sublime Text, I wanted autocompletion for Ruby there. There was a plug-in that claimed to provide it. I don't know if you can see very well, but in that little circle, it says Ruby, it claimed to provide this. I tried everything to set that thing up and it just didn't really ever quite work. And it didn't work with JRuby, which was something I had started getting really interested in. So I dug a little deeper. What would it take to make this kind of tool on my own? Well, everything I read said that to do code completion, you have to do type inference and doing type inference on Ruby was basically impossible. A lot of quotes like this one, but I'll read it because I really like this quote. Metaprogramming makes the type inference problem equivalent to the halting problem. For anyone that doesn't know, the halting problem has been proven mathematically to be basically, or to be impossible. But I'm stubborn. So I kept digging around and I found this tool called R-Sense that did everything I wanted to. It was created by Tomohiro Matsuyama. Unfortunately, it was abandoned. The last commit was in 2010. And when I got interested in it, it was hard to install. It only worked with Ruby 1.8 syntax and it was using a really old version, like 1.3, 1.4 of the JRuby jar to do the parsing of the Ruby source. So being naive and really new to programming, I thought I could just swap out the jar file and pop in the new JRuby and everything would work. You may not be able to read there, but bad poker face guy is saying, of course I knew that wouldn't compile, but I had no idea. So I found out about a program called Google Summer of Code. It's a program Google does every year where they allow students to work on open source projects and they pay them a really nice stipend over the summer to do that. And I found out that JRuby was gonna be a participating organization. So I went out to my local community college, signed up for a class so that I would qualify then I applied for the program. That summer, Tom helped me fix over 800 errors and 100 failing tests. I learned a lot, we got everything working and it was really awesome. So how does a noob like me manage to work on something that is pretty hard stuff? Well, there were a few things that kind of had going for me. I was comfortable with moving really slowly, one bug at a time, one failing test at a time. You can't hear it, I'm sorry. But reading the source code, just diving in there, not understanding it and being okay, being comfortable with not understanding what you're looking at and gradually a picture will start to emerge. Now when I say debugger, a lot of you probably think of these things. But I found that a debugger that let me step through the code line by line and learn how it was executed, how it flowed really allowed me to get my head around what was going on. Reading papers, of course it's gonna be full of lots of words you don't understand, lots of math you don't understand, maybe I didn't. But gradually doing all these things, a picture will start to emerge. These are a couple of the papers that I read many, many times before I really understood how arson's worked. The most last thing though was being really stubborn, just being determined that this thing was gonna work and I didn't care how much time I had to put into it. So, at the end of the summer, had squashed a lot of bugs, fixed a lot of broken tests, things were working, but had you looked on rubygems.org you wouldn't have found a way to install arson's. I had some questions before I released anything. Why, since arson's had been around for so long, had it not been more widely adopted? This is cool stuff, it's a cool tool. People had a really hard time installing it and it was written in Java, so people, rubious, the people the tool was made for, probably maybe didn't feel comfortable contributing to it. So it needed a better install story. Now, over a year later, if you look, you will be able to gym install arson's and then install the editor plugin. Currently there's three and we'll talk about those later, but there should be more. And it'll work for either MRI or JRuby. Arson's, the gym uses Spoon, which is based on POSIX Spawn to launch a JRuby process. This makes it possible for you to install essentially Java code base with MRI and still run it. This then runs the arson server, which communicates between whatever editor plugin and arson's core, which is just a very thin wrapper around the arson's jar. You can see in this diagram how that works. You gym install arson's, run arson's start, Spoon then spawns JRuby, which starts the server. That talks over HTTP to whatever editor plugin. Then it makes Ruby method calls into arson's core, which calls into arson's jar. Arson's is implemented with a variation on the Cartesian product algorithm, named after Descartes. CPA was first implemented by Ole Agasson for the language self, and I referenced that paper earlier. There is also a paper describing a project called Ecstatic, and the algorithms described in the Ecstatic paper are almost identical to what is actually implemented in arson's. If you search for it, you'll be able to find it if you're interested in reading it, or I'm happy to share links to those papers. So how does arson's implement CPA? First, code has to be read in and parsed into an abstract syntax tree, which I'll call an AST hereafter, by JRuby parser. An AST is just the representation of source code in a tree-like structure. We've got some very simple Ruby here. It's just an assignment of the number one to a variable A. If we parse that with JRuby parser, we get back something kind of like that. Here's a little bit slightly more complicated. Bit of Ruby. And if we parse that into an AST, then you have a structure that's just more suitable for processing. So here we've got the if node, like in the code, and that's got some branches. It's got the conditional, and then it's got what happens if it's true and what happens if it's not true. JRuby parser is a library that originated as it was extracted from JRuby and it has special methods useful to IDEs and tools like R-Sense. One of the really good things about it is that it tracks source code position. So R-Sense walks the AST after it's been parsed out by JRuby and builds up a graph of nodes using an abstract interpreter. Walking the AST, it uses something called the visitor pattern to do that. Every node in the AST will implement an accept method and which then will call the specific visit method on the visitor that's passed in. So here is some code from R-Sense called create vertex. This is creating a vertex in the graph that R-Sense builds up. And you can see it calls accept on the node object and then passes itself in. Then here we have one of those visit methods, this one for a call node and you can see that then it does things specific to a call node like a call node will have a receiver. So it creates a receiver vertex by passing back to the create vertex method we just saw. And this is how it walks through the AST. ASTs are pretty awesome. One of the things that I did while working in the Google Summer Code Project was took some things out of R-Sense and ported them back into JRuby parser. One of them was the ability to do a semantic diff. So a usual diff like what GIF does is text-based but a semantic diff is actually looking at what the code is, what it represents. And so you can do some smarter things with it. I made a tool from that called SmartDiff and look at a little GIF here. The left is the older version of the code and on the right is the new version of the code. The stuff in red is stuff that's been removed, stuff in green is stuff that's been added and the stuff in blue is stuff that didn't change but might have moved around. Now in a text-based diff it usually can't tell if you move methods around inside your file. It will think that that's completely new code. But with a semantic diff it can and it can track it so that when I click on one side or the other it will move the other side to line those methods back up. Abstract interpreter. So while walking the AST R-Sense uses what's called an abstract interpreter which I like to think of as a virtual ruby. It goes through each line of code as it would be executed but it doesn't actually execute it, it just understands it. It gets information about it. Then it uses what's called constraint propagation to take the types that it knows and move those along the graph to the objects in the graph where the type is unknown. Here we've got some simple code x equals one then we set y equal to x then we set x equal to hello. Because we know x is a number or one is a number we assign that type to one that propagates through the graph down to x and on to y. Then because we also set x to hello a string we propagate the string type into x. R-Sense uses some built-in methods to stub out the core library and the standard library. This gives it information that would be missing otherwise and so it gives it a base to build itself up from. This is what those stubbed out methods look like. You can see above them there's some type annotation and we'll talk about that a little bit more in a minute. So all this would be almost enough but Ruby's dynamic metaprogramming feature it means that there's a small segment of valid Ruby code that R-Sense can't do type inference on. It's very small but it does exist. This quote comes back. But R-Sense does have a solution for this too. It uses an annotation resolver taking a special type annotation syntax in the comments and it uses those to get around those rare edge cases where it otherwise wouldn't know and because it does that it prevents bad results caused by those from bleeding into the rest of the results. For the most part it's only necessary to have those in the built-in stubs but what I wanna do is move it to a syntax like Yard uses, the documentation tool. It has a great type annotation syntax that many of you already know and so you could improve your documentation and get better type inference at the same time. So that's a goal. So here we have a module with a class inside it. There's a method, it takes a variable and it yields a block to that variable. So we'll pass in an array with a string inside and a number and then we'll pass it a block. So we'll get the first element of that array and we'll see that it knows it's a string and provides us a string method. We'll get the second element of the array so I start to type it'll clear out the and we'll get ABS which is a fixed num method and if you wanna take a look for yourself at that demo you can at that web address. So where does our synth stand today? There are currently three editor plugins working known to be working, one for the Atom Editor that I wrote, another for the Sublime Text Editor that I also wrote and a third for the Tech, for Tech's Mate 2. I have currently very recently switched to Emacs and so now I have motivation to make an Emacs plugin. I'm really looking for somebody that wants to take on making a Vim plugin. It works great currently for Sinatra apps and when you're working on like a gym or a library but it does currently fall down on Rails apps. It runs out of memory. Rails plus all of its dependencies is just too big for the graph but I've got a solution for this. We could do some kind of caching of the graph and that would be able to get around, work around that limitation. It just needs to be implemented. So I've got a big to-do list here of Vim and Emacs plugins caching the type graph, switching to yard style type annotations. Currently, Arson's, the Java code actually has implementations for find definition where and type inference beyond just the auto completion that you saw demoed a minute ago. So those just need to be exposed to Ruby and to the command line or the Arson server so that editors could take advantage of that. Then on top of it, we could build error detection tools. There's a project called laser that also does type inference and has a lot of error detection capabilities built into it that could be ported over to Arson. I would just say just use that project but it doesn't look like it's being actively maintained anymore, although I could be wrong about that. So we could port those over to Arson. It's probably pretty easily. Automatic refactoring tools. I think these would be really exciting to have in Ruby. So Arson's needs you to use it, to help finish it and to build on top of it and to make a plugin for the editor of your choice. You can find out more about Arson's at arson.github.io and thank you very much.