 So I was trying to come up with a topic for Ruby Conference. This is actually the first Ruby Conference I've been to. And so I was really looking for an excuse to go and take a few days off from work and come down to Peninsula. I'm a local. And so I thought, well, I could talk about my own stuff. But that's not really all that interesting, just to talk about that. So I decided to look for something that other people were doing similar to what I'm doing, which is using a domain-specific language to do code generation. And not just to do code generation to SQL or JavaScript or Ruby code or something like that, but to actually generate native code. And so this is something that might be used in embedded development or in numerical computation or graphics or something that requires the performance of native code. So to my surprise, I did find something very interesting that I hadn't known about. And so I'd like to start off talking about that. Then I will talk a little bit about the work that I'm doing. Then finally, the final two topics, I'm going to talk about two hypothetical projects. They're really more a desire of mine to have two tools that would do in a better way the sort of thing that I do all the time in a really crappy way. And maybe the result of this would be somebody will raise their hand and say, yeah, I've got that. Get this gem. That seems to happen quite a lot. Or if that's not what happens, maybe somebody will say, yeah, I'd like to work on that. So I'm hoping to get some interest on that. OK, so the first topic is called RGEN by a fellow named Martin T. De. I think I'm pronouncing that right. He lives in Germany. He works for the Lear Corporation. And he's actually using this to generate production code for BMWs. It's actually running inside of at least some model of BMW. And it's a very recent project. He's been presenting it and writing about it recently. And it's a gem. It's got a good web page with some presentations if you're interested. So what is RGEN? It's actually four DSLs. There's a metamodal builder and a model builder. And the transformer, I'm not going to talk too much about that. And then there's a templating DSL, which is based on ERB. But there's some interesting stuff that's added to it. It's not just plain ERB. And that's what you use to do the code generation. So the metamodal builder. So through this example, we're going to keep using the same example, which is a state machine. So we're going to define what a state machine is. Then we're going to build an instance of a state machine. Then we're going to generate some code to actually execute the state machine. So this is the first step. We're going to define what a state machine is. So we're working at the meta level. So we're going to define this module. That's just sort of the group all of our classes together. Then we have this nice DSL. We define a state machine metamodal just by defining a class called state machine. And then we can define an attribute of that class. In this case, it's a name attribute and it has a string. Its value is a string. Same thing for a state. So you all know what a state machine is, right? Basically, it's a model of computation. You've got a bunch of states. You've got transitions between the states. When an event happens and you're in state A, maybe you go to state B, or you go to state C, or whatever, depending on which transitions there are. So the states are related to the state machine in a containment relation. It contains many relations. And you define that relation in this DSL. And what that is saying is that state machine is going to have an accessor called state, which you can use to access a set of states. The elements of that collection are the state instances. And the state instances, the last bit on that line there, state machine, says that those state instances can get back to their container, the state machine, via that other accessor. So they have an accessor that goes back the other way. So it's a bi-directional relationship. Then there's a composite state, which is sort of a sub-state of a state. And then there's transitions, which also belong to a state machine. A state machine contains many transitions, and that's the same kind of bi-directional relationship. And transitions are also related to states. A transition has a source state and a target state, and it's a mini to one relation. A state may have many outgoing transitions, as well as many incoming transitions. So what can you do with one of these metamodels? Now this is actually one of the really interesting features of our gen, is that it reaches outside of the Ruby world, because you have to to talk to embedded systems designers. They don't use Ruby. They don't use Eclipse or UML or lots of other things. So you can generate UML from one of these metamodels. You can generate something called eCore, which is apparently the kind of the native concept to the Eclipse modeling framework. Or you can generate a model. And I'll show you that third kind in a minute. There's actually two ways to get a model from a metamodel. As you might imagine, this is Ruby, so just instantiate the class. Really easy. You can instantiate a state machine and give it a name, instantiate some states, instantiate a transition and just set these accessors. So when we, for example, when we define this transition down here, we say that its source is the off state. Well, then that T1 will have an accessor called source, which will get us back to the off state. But there's an easier way and a more clean DSL way to do the same thing. So this is the DSL that you get simply because you've defined those metamodels. So we have this DSL where we can say there's a state machine with a certain name. It has a state called off. It has a composite state, which has these two substates. It has these triggers. You can think of triggers as events in the system. They'll be related to transitions. When a trigger happens, a transition can go from one state to another. And then we can define transitions with all the various parameters. So just to focus on this a little more closely, because we had a metamodel that had these things in it that had a way of containing transitions within a state machine, that gives us a way of a DSL method to actually add a transition to a model. So it uses that metamodel. You can maintain the metamodel and then maintain the code that uses. You can change the metamodel to add a new method. In fact, you can also define methods that call these. I'll come back to an example of that in a minute. Two things of interest here. One is that these models are actually round-trippable to the DSL. So if you want to save your model as DSL, you can instantiate the model and you want to write it out to disk, you can write it out in the DSL representation rather than in YAML or Marshall or whatever. So that's kind of nice. It gives a nice native way of representing it. Another thing to point out is that so far, everything we've done, it's just kind of an abstraction. It's just an entity-relation diagram, basically. We have many-to-one relations. We have containment relations. We haven't actually made the machine run. It doesn't actually do anything yet. So that's really the next step. We're gonna take a look at the templating DSL. So it's based on ERB, but it has these really two interesting additional features. And maybe it's used in other places where people use ERB, but I've never seen this before. So within your templating language, you can define a new template and then in another place in the templated file, you can expand that template using some parameters. So let's quickly look at an example here. There's a lot of code here, but briefly what this is going to do, and I'll show you the output in a minute, is it's going to generate a case statement that handles the transition behavior of our little state machine. So if you look at the lower part first, where it says define case for state, what that's doing is it's defining a template called case. And the four is saying that this template makes sense in the context of a state instance, a state model instance. So whenever we use this case thing, we have to be inside of a context where we know what state we're talking about. And so what the template is going to expand to is the code inside there. It's basically if you, it's C code, so think about a switch with cases inside of it. This is just one of the cases inside the switch. So we're going to emit a case S underscore then that qualified name thing. Well, qualified name, we haven't seen that yet. That's actually a method that's defined in terms of name and I believe maybe the state machine name as well. Anyway, it's a method that you define on top of the generated method of the DSL. So the next line says iterate over all out trans and all out trans is another one of those methods that's defined in terms of the reflection API of the state machine. So iterate over all of those and emit an if statement that checks whether the trigger is the trigger of that transition or not. So basically look through all the transitions and see which one matches the trigger. And if that's the match, then set the state to the state that's defined by the target of that transition. So let's, well, okay, before I move on to the output of that, take a quick look at the sort of the global template up there, which is going to output the whole function. So that outputs a function called trigger with the switch and all of these, the switch, which is gonna contain all of the case statements. And the way we do that is by expanding the case template and that for each, so that's something new. That means that we're going to iterate over the value of that expression, which in fact turns out to be a list of states. Don't worry about why we're rejecting stuff. Okay, so let's see what the output of that is. It's a nice little C function. It takes a trigger as an input and then we've got these three cases, one for each of the three states that we defined and it looks at the triggers that we defined and goes into whatever state matches the trigger. So I think this is kind of nice, well for two reasons. One reason, as I said before, is because it works well with external tools. You can talk to UML or whatever. Also, I think this way of defining and expanding templates within your templating code, it looks promising. Maybe it's been done in other places but it looks very useful. Okay, so that's it for our gen. Unless anybody has any quick questions, I'm gonna move on to the next topic. So for approval, why not use an Espen switch then? Why are you using Elsie and Elsie and Elsie? Good question. It's actually not my example, it's Martin's but you could probably do it another way, sure. So I'm gonna go on to the next topic, which is something that I've been working on for a while but first a little background because this is actually a more narrow focus than the previous subject. The Argin tool is really very general. It's a very general templating tool but it doesn't have much knowledge of a particular subject domain. And now I'm gonna move on to a tool that has a lot of knowledge of a particular subject domain. It's actually a kind of state machine related topic but they're state machines of the very specific semantics. So I have to explain a little bit about what that is. So first a hybrid system. This is a concept from kind of engineering and applied math. Hybrid system has continuous dynamics like a falling ball. And if you remember calculus, you can drive a formula for the vertical position of a falling ball. That's too simple though. That doesn't really capture a lot of real world phenomena. Hybrid systems are a lot more than that. There are things like this where we have continuous dynamics of falling ball plus discontinuity. When the ball hits the ground, velocity goes to negative of its velocity. It's possible to come up with a formula for that but it's much harder to write down than what I wrote down before. But that's still not enough to really cover the kinds of real world phenomena that I'm interested in. So what I'm really interested in is continuous dynamics plus unpredictable discontinuities. So not just discontinuities but discontinuities where you have events that are happening that you cannot predict in advance and they change the continuous behavior in a big jump of some kind. Like shifting gears in a car for instance is a good example of that or any kind of communication based thing tends to happen that way. So you have discrete events feeding back into a continuous system, changing the state of the continuous system. Okay so there's really quite a lot of these when you start thinking about it. Cars are in more ways than one. The engine itself is if you've got gears because which gear you're in changes the continuous dynamics between the wheels and the engine. UAVs, unmanned aerial vehicles, robots. Packet networks are too. They can be modeled as hybrid systems because you have continuous time and discrete events happening at different points in time. Also cellular and chemical processes can be thought of as hybrid systems too. So just to be a little more precise what's really hybrid about this? There's really sort of two kinds of categories. One is digital control in an analog world. The other is discretized continuous phenomena in a continuous world. And an example of that is a ball bouncing and that's really a continuous phenomenon if you model it at a fine enough level if you really get down to the compression of the rubber and all of that. But we don't want to do that mostly. Mostly we just want to say okay the ball hits the ground and it bounces. That's enough. We don't have to model it any finer than that. So we discretize those kinds of events. And the same is true for an engine with gears as well. So why are we doing this? Why are we even interested in talking about this? Well it's kind of an interesting field because there's not much theory there. You can't really prove theorems about what kinds of systems are solvable in one way or another. They're not like ordinary differential equations where there's really a lot of theory. And it's a new field. And we get to play with cool toys. These are tuning automated vehicles. This was a demo in San Diego in 1997. The separation between vehicles is I think three meters but they were technically and safely capable of two meters. You'll see in a minute that okay there's somebody holding the steering wheel and then letting go of the steering wheel. So these truly are automated vehicles. They're following each other. They're using wireless communication and as well as radar to maintain steady safe distances. There's a radar, actually. There's another example of the same kind of thing but using much heavier, more dangerous objects. I don't wanna be in those trucks. They don't let us do those on highways. That's down at Crow's Landing, if you know where that is. Out in the Central Valley. It's an old air base. Yeah. Yeah, well, we were doing it in this country in 1997 but it didn't go very far. It kind of lost popularity. And not only the speed control. There were algorithms for merging into a platoon, merging out of a platoon, lateral control. The interesting thing to me about all this is that the control system, the control software for these things that we just saw was designed and validated using a programming language called Shift. Shift was invented at California Path at UC Berkeley. It's where I work now. That was quite a while ago. Let's come back to 2009. Things have been kind of quiet in this field but it's starting to catch on. The National Science Foundation is spending $30 million a year now on what they call cyber physical systems which is really hybrid systems, a little more general perhaps but it's the same basic idea. So we're programmers. What do we do to get involved in this if you're not an engineer or a scientist? Well, what's underlying all this is a model of computation. Hybrid system is just a dynamical system. It's just a system with state evolving over time. But you can model it with an automaton, not just a state machine. It has to have continuous behavior as well so we call that a hybrid automaton. And so here's an extremely simple one. It's got one state called freefall and the equation should be pretty familiar to you. The derivative of velocity with respect to time is negative 9.8 meters per second squared so that's gravitational acceleration. Ndy dt, the change in position is velocity. That's just a Newtonian mechanics. Bounce is a transition that happens when y is less than zero so when it hits the ground and it's falling you reset the velocity of the negative velocity. So that is actually what we were seeing a minute ago. We were seeing those bouncing balls. It's the same model of that behavior. But it's not just individual, it's networks of these things. So for example, you have a thermostat in a house. The thermostat has an output to the cooling system, the heating system. The house evolves over time and has a temperature that's sensed by the thermostat. So we have data flowing in a feedback relationship between these two components. In each of these components, we can model perhaps as a hybrid automaton. So we have a network of hybrid automata. But it's not just networks, it's dynamic networks. So this is a simulation of a highway. The arrows are telling us something about the internal state of the control system. If you think about it this way, if you're driving, you have to pay attention to the guy in front of you if you're gonna slow down, when he slows down. If you wanna change lanes, you have to pay attention to the guy before you on the left, behind you on the left. And let's go back and see that again. So basically you have to pay attention to the stuff around you. So you have to maintain these relationships. Whether you're doing this attention via visual perception or you've got some kind of wireless communication or whatever, a simulation needs to know these relationships. But the relationship's changed over time. As one vehicle travels faster than the other or changes lanes or whatever. So we have a dynamic network of these data flow relations. So back to the question of how a programmer gets involved in this. We've seen these systems get more and more complex. Close form solutions are just not possible when you have this many different things going on. Imagine a whole freeway of cars. You can't write down a formula that says that at time T, my car is 60 miles down the road or whatever. So you have to do this via simulation. So engineers want simulation tools to tell them answers. They want to tell them whether a controller works, whether it has good properties, do the cars crash. And in fact that's what we did in 1997. We sort of proved using simulation that these controllers were string stable which means that the lead guy breaks. Number 10 isn't going to get in the crash. And also you want to see the emergent properties of networks because you've got these big networks moving around, you might have flocks evolve. And we know that on freeways traffic jams evolve. Traffic jams don't happen because there's something in your car that says that you have to make a traffic jam that it happens because of the emergent behavior of all the drivers. So as programmers, let's write software that designs and executes these hybrid networks of these dynamic networks of hybrid automata. Why is it a good idea to do this in Ruby? Ruby's slow. Engineers don't use Ruby. How do you convince your boss to use Ruby? Well, thank you, Mat San, for making a wonderful argument this morning. Ruby is perfectly suited to code generation and especially internal BSLs. So if we can do that, why not write a high performance language for this abstraction that we're interested in? Okay, so here's an example in that language. It's called Redshift. This is the bouncing ball class that was animated on the screen a minute ago. It's actually pretty simple. We define some flows and those are just those differential equations. And we define a transition. Now, this is a one-state system. So implicitly, this is all being defined on that one state. But if we had multiple states, we would be saying the flow for states such and such is this, the transitions between states such and such but in this minimal case where there's only one state, that's all implicit. So we have a transition. We have a rule for when the transition happens. That's called a guard. And we have a reset, which is a discontinuity. The velocity changes in a discontinuous way. And that little action there, that's just something that we can spit out, output to the terminal or whatever. So let's instantiate one and evolve our world for five seconds. Okay, so we run the program. It takes a little while to get started. And so this is basically the data that we would see that corresponds to the falling fall animation that I showed a minute ago. And it bounces when, oh well, bounces when the position was less than zero. So what's happening when we do this? Well, Redshift has this co-generator that turns this into a rather large C library. It builds a Ruby extension. So this all works within the Ruby interpreter using the usual xconst.rb and makefile and all that, mkmf. Then it loads it. Then we had that line that said evolve. And what that does is at each time step, which I guess in this case is probably a hundredth of a second or something, or maybe a tenth of a second, it numerically integrates those ODE's. If you know any numerical integration, it's a fourth order Rungekata integrator. It checks the guards whenever it needs to to see when something has happened, when the ball hits the ground, and then it applies the transitions when those guards are enabled. As a DSL, this is really pretty standard stuff. If you've seen other DSLs, it uses the same kinds of techniques like method missing and instance eval. So I won't say too much about that. One thing you might have noticed is that the equations and formulas are written as strings rather than as Ruby code. That makes it much easier to turn it into C code. It doesn't look as nice. It would be nicer if we use some kind of Ruby parser thing and so that we get parse trees. So that's the DSL side. The code generation side of this is based on a gem called Cgen. It's probably a little less general purpose than Rgen, but it has some nice features. It's optimized towards defining user-defined objects that have a struct, a C struct attached to them. If you know anything about the Ruby extension interface, those are of a type called Tdata, and you can build up that struct and do whatever you want with it. But there's ordinarily no Ruby access to the members of that struct. Ruby, it's a blob as far as Ruby is concerned. You have to define methods in order to access the struct members. And that's what Cgen does. As you define members for a struct, it gives you Ruby accessors. It does the type checking and converting code. It does all the mark and freeze stuff necessary so that if you have links between objects, it correctly marks the linked objects so that it doesn't get garbage collected. It does the marshaling code so that you can save these things to disk, allocation, initialization. Another interesting thing that it also does is inheritance because normally, if you have a Ruby class that has a Tdata associated with it and you inherit from that, there's nothing in Ruby that's gonna make inheritance work. So we have to sort of manually, I mean, as far as Cgen does this, inherit the struct members in the subclass. Now, it also allows you to write inline C code. So it's kind of like Ruby inline. You can use some simple templating features that put stuff in the header, put stuff in the declaration area, add an argument and so on. So I'll show you an example of this in a moment. And at runtime what it does, it generates the source files, builds an extension, loads it. And it's very careful only to write what it has to write so that if you run the same thing twice, the second time it'll go very fast. So here's an example where we build a complex number class. So it's very simple. You just include the C shadow module and then you're allowed to define these shadow attribute accessors. In this case, we have an accessor for the real part and accessor for the imaginary part and those are gonna be stored in a C struct as two doubles. And we're gonna define one C method just as an example, the absolute value function. And this shows that you can include header files as well. You can really connect to just about any library you want like the new scientific library, GSL if you wanna use that. And this is a very simple example because all we have to do is do one little formula and return it. And so that'll be a return to the calling Ruby code. Okay, so let's see this in action. We're gonna load it up in IRB just so we can play with it. So it took a little less than a second to build. Now we're gonna instantiate a complex number and take it to absolute value. Yeah, that looks about right. And leave IRB and enter IRB again and see how long it takes. This time it took only a quarter of a second rather than a second. So that's because all the files had already been generated and it checked that it didn't need to generate anything new and it just loaded the extension. We can go on and add some stuff to this. Here's a method that destructively scales a complex number by a certain scale factor. So the idea is that if you have a complex number you can scale it by three or something. And what this shows off is that there's a nice little mini DSL for the case where you pass a C array of arguments to your function. I don't know if you've got any familiarity with the Ruby extension API. There's three different ways to do it. You can have a Ruby array, you can have a C array. And when you have a C array you have to use this function called RBScanArgs which is a little tricky to get right. But this really gives you a nice DSL that lets you specify which arguments are optional, what the defaults are, whether you should type check, and also whether it's something required, a REST argument where you just take all the rest of the arguments and stick them in an array or whether there's a block. Okay, so here's just a sample of the output that comes out of this thing. So the one on the top is the setter method for the imaginary data member, struct member. And so it looks just like what you would write if you were writing this extension code yourself pretty much. Then there's the absolute value. So it's really, and it does nice indentation and so on. So the generated code is pretty readable. I want to point out though that code generators can kind of get a black eye from some people because you generate code and then who wants to work with the generated code? That's not the intent of this at all. The intent of this is that you generate the code and kind of forget about it. So you always stay at the source end of the code generator. You don't generate stubs and can start hacking on the stubs. Okay, so that was a little diversion into the code generation end of Redshift. So now I just want to come back to a few generalities about the project. So it stays object oriented like Ruby. And there's actually, as I mentioned, there's inheritance. There's actually a fine grained inheritance of the flows in the transition, which is kind of neat. We can use C numeric libraries. There's a product called Simulink, which is part of MATLAB, one of the MATLAB tool kits. And it's one of the widest used simulation tools for engineers. Inside of General Motors, it's their standard. They use it for everything. Simulink puts blocks on the screen and you put wires between the blocks and each block does some kind of processing. That's called a Dataflow diagram. Basically, Redshift does the same thing, but without the GUI. But it does more, Redshift does more. Simulink doesn't do dynamic networks. Once you put the block diagram down and you put the wires between the blocks, that's the way it's gonna be for the length of the simulation. Things don't move around. So that's why actually Shift was invented. Shift was invented because you couldn't use Simulink for this kind of platooning vehicles where you get a new vehicle that tries to join a platoon and changes the relationships between vehicles. Like Erlang, it's got cue-based communication between components and it has the same kind of pattern matching feature. But before you get too excited and think, okay, this is competitor Erlang, it's not, it's a simulation language. All of the current concurrency that's going on between these different components, different hybrid automata is simulated concurrency. It's not real concurrency. But there is some hope that maybe the semantics will let us execute it on parallel processor systems. So just a brief history, been around for a while, mostly in-house stuff, some stuff with General Motors. Status is pretty good, performance is close to Simulink. And I'm really hoping that this actually goes somewhere with the recent interest in cyber-physical systems. So that 10 minutes left, so I wanted to get on to the last two topics. Which are kind of hypothetical. Well, before I do that, any questions about Redshift or anything? Okay. So what I want, because I've needed these things over and over, is a way of doing animation from Ruby or from anything else for that matter. And a way of writing network packet handler code that's efficient yet doesn't require you to write C code. And the motivation here is that, when you use Ruby, you've got the garbage collector, and your animation does a hiccup every now and then if you're generating a lot of Ruby objects. And for network packet handling, you don't want to be using Ruby unless, I mean, if you're really serious about performance, you've got to use C. And as a bonus, you get this, if you can generate, if you can generate some kind of a C library, that becomes a API that another C program could use. So if you work in an organization that has a lot of C programmers, they can use the output of your code generation too. So what do I want from this thing? So for handling network stuff, well, my idea is that, okay, we've got a lot of things, we've got like event machine, but event machine just, it does a great job of multiplexing data over sockets in one thread, but it hands the data off to you and then what do you do with it? You write Ruby code to deal with it. If you need to do some kind of complex filtering, do pattern matching, extract things, merge things, transform things. I'm not sure there's really a good efficient way to do that in Ruby. So I'd sort of like to see if there's a way of writing a DSL that lets you describe those processes, to specify transformation rules and filtering rules, and would somehow generate efficient code to do that. And I'd want this to be, to make Ruby more attractive for use in embedded systems, like, you know, gumsticks, single board Linux computers that weigh as much as a teaspoon of water and are the size of a stick of gum. They're very cool, you can put them in UAVs, fly them around, but it's hard to run Ruby on them. I've done it, it works, but you're gonna get some arguments from people that, you know, you're using up too much memory. So actually, if memory isn't the issue, there's still the performance issue, which is why I'd want something, some kind of a DSL for more efficient packet management. The other one is the animation DSL, and the reason I want that is that, you know, I'm doing these simulations, and mostly I use TK. Well, you know, TK is sort of the default, but it's kind of crappy in some ways. So I really want to be able to define shapes and move those shapes as data comes in from a simulation, or for that matter, as data comes in from some kind of a real world system. Whatever, there's some source of data that's supposed to drive things around somehow. And it would be nice if the back end were switchable too, it'd be nice if you could say, well, I've written this specification of how the animation is supposed to work, but you know, today I want to do it in OpenGL, but tomorrow maybe I want to do it in an HTML canvas or something. And it would be nice to have that same specification work for both kinds of things. So actually, I did a little project called TKAR, which was a very poor version of this. It was based on TK Ruby, and so it's got the basic problems, but this really needs to go to the next stage. So if anybody knows of something like this, or if they're interested in working on this sort of thing, let me know. And that's actually it, unless anybody has any questions. So, not in the animation, but in the other one. Yeah, Bert looks interesting, but your mic is off. How did that happen? Hello? Testing? Yeah, this, is that working now? Yeah, okay. So Bert, yeah, that was a very interesting talk. And if you're sort of doing Greenfield's development, if you're developing a system to use Bert from the beginning, that's great. But there's a lot of cases where, particularly in the mobile wireless world, you go to these meetings and conferences, and everybody's got their own protocol. There's all these protocols out there. Nobody knows about Erlang. Nobody knows about Ruby. Nobody knows about Bert. There's all these things that are defined and you need to deal with them. It's very similar to Arjen, and just sort of as a volunteer, I'm probably gonna be doing something with Arjen and the VHDL, or at the end. Fantastic. Yeah, so are you in contact with Martin or? No, I'm actually, I just discovered it, sorry. Oh, okay, great. I'm doing it for volunteers. Cool. Yeah, I think I may be using Arjen as the next generation backend of my own work as well. It's very clean and generic. Okay, thank you very much.