 And this is Bob Allen, and we're going to be giving a talk that we retitled App Engine and Dubious and new Google APIs. Oh my. So I'm Nick Howard. I'm an avid Rubyist. Dubious is one of the open source projects that I can contribute to. So I'm going to be talking a bit about that. I'm Brooke Bobcat on Twitter and pretty much everyone else. And they're working at Gnip in Boulder, Colorado. Hi, I'm Bob Amon. I work at Google in developer relations. My primary responsibilities are actually social web and buzz, which is completely unrelated to what I'm going to be talking about today. John Woodell was originally scheduled to give this talk. He really wanted to be here. But unfortunately, he had some personal commitments, so he wasn't able to make it. So we're going to be giving the talk instead. So I'm going to start off with a quick App Engine update. There's some new stuff coming up in App Engine that we wanted to tell you about, a lot of which is going to be very interesting if we think to the Ruby community. So the three main new things are the channel API, the map root API, and the in-space API. Channel API is really interesting. I'm not sure how familiar you guys are with App Engine, but the way App Engine currently works is we have a time limit on all requests. So you can't really have a request that lasts longer than 30 seconds. So if you're trying to do stuff like comment or long polling, that can be a real problem. So we're introducing the channel API as a way to deal with us. It's an asynchronous server client communication. It's bidirectional. And we actually use the exact same technique in the Google Talk client. So we're making that available to basically everyone who is using App Engine. It's actually pretty easy to get up and running. I'm showing the Java interface here, but it's basically the same idea if you're using Ruby, because you're just going to be dealing with the Java and the interfaces. So you have three methods that you have to define, create channel, send message, and parse message on the server side. Once you define these three messages, you can basically transmit whatever messages you need to the client directly with the browser. And then on the client side, you have the JavaScript API. You get yourself a socket. And then on the socket, you can define events on open, on message that will handle the messages that you're transmitting back and forth to the server. And then there's the send call as well, the send message back to the server. So that will let you do a lot of really interesting real-time stuff that isn't currently possible in App Engine. And that will be coming soon. We also have the Mapper API. How many people here are familiar with MapReduce? OK, fair number. The MapReduce API is the map part of MapReduce. It is an open-source project. It's actually live right now. You can go to MapReduce.app.com. So maybe we actually have internet connections. And you can see the project. There's a branch for Python and a branch for Java. And both of those are available now. We will be adding in the shuffle and reduce steps of the MapReduce algorithm at a later date. And that is coming soon-ish. The whole project is built on task use. And so when you go to install the Mapper project into a Map Engine app, it's basically going to take advantage of all the capabilities that are already in App Engine. So that's actually a very interesting project. And just as a diagram, this is the architecture of how MapReduce works. You have a map step, shuffle step, and a reduce step that allow you to do very massive parallel processing on distributed data sets. And the part that we currently have available is the map step, which is basically the first part. And in terms of practical usage today, probably one of the better things that you can do with it is basically changing your schema on the fly. So the data store for App Engine, it's essentially a schema list. So you can have a schema change after the fact. Then all the old data will still be using whatever your old setup was. So now if you need to update all that data to the new schema that you're using, you can do a map process and run a bunch of jobs and workers to realign your schema to what it should be. And then we have the namespaces API, which is a really interesting new technique. We actually kind of did something already. We've had it for a while in non-cache where you basically have been setting the namespace on your keys. But we're also introducing that for the data store and task queues and giving you a really clean way of introducing those namespaces. So that allows you to do data isolation. So for example, you can compartmentalize information based on what user you're currently looking at. You might want to segregate admin data away from the rest of your application. Potentially, you might have multiple environments. So like with Rails, we have a production environment or a staging environment or a testing environment. Namespacing will allow you to run everything against the same data store, but then segregate the data based on namespace. And potentially, you can even run multiple applications in the same environment without having them colliding with each other. At this point, I'm going to hand it off to Nick and he's going to talk about Mira and Gubius. All right, so as Bob was talking about App Engine, so currently there are a number of people running apps on App Engine using J-Ruby. And it works pretty well, but it does have some limitations because J-Ruby is more dynamic than running something in Java and it has a larger runtime than running things on Python. And it's not supported as well by Google because it's not one of the primary languages for the platform. It also uses a lot more CPU time, which since the way App Engine is priced, that means that it costs more to run J-Ruby instances. And if you want to get more performance, you need to write Java, so it costs more money. The kinds of things that cause John World to want to start using Mira as a language to build web applications with on App Engine or situations like this where you have these large spikes and so you have to spin up a lot of instances and if they're J-Ruby instances, you need more instances because of the processing. And so with Mira, it's faster and it's a smaller runtime, so it's less expensive. And as I said, on App Engine, CPU time is money, so that's kind of important. So the way that you used to optimize your J-Ruby App Son App Engine would be to, while you write an app in Rails, then you'd take the parts of it that were a lot or slow and then you rewrite those with Java serverless and maybe you'd have to host them on a separate app or something like that, so it's the same database and it's kind of complicated and it kind of sucks. And so that's where Mira and Ruby is coming from. So Mira, which used to be called Duby, is, it's a pretty cool language. Mira means Ruby in Japanese. Charles another wanted to create a language that looked like Ruby but was statically typed and compiled to JVM bytecode like Java does and so that's, you're not sure what Mira is. So it uses Ruby syntax, literals, and some typing stuff but it behaves like Java so it has a class model that's more like Java. And you can extend it through MapRos and you can write MapRos in J-Ruby or in Mira. So it has optional arguments, closures, literals, string interpolation, et cetera. So for example, here's a naive implementation of fit and option Ruby. In Mira, you just have to add type declarations to it and it does a lot of inference too so you don't have to specify the type of every single variable on a signet. It will do some of that for you. It also has MapRos so you can do things like, you can iterate over a collection using each or there are a number of other helper methods like that and I think as the language gets more involved there's gonna be more support for writing your own MapRos and including them in your apps. So dubious, dubious is a web framework in Mira. So it's high performance, lightweight and it's a lot like Rails. So since it's written in Mira, it's about as fast as Java can be or faster than some, I mean like if you're using a large Java framework it's faster than that so if you're using like Pygoremi or something. On App Engine, its instances spin up a lot, spin up very quickly so if someone hits a new instance with a request, because what App Engine will do is if you're getting low it will start automatically spinning up instances for you and so the instances can be ready that much faster which is very good. It's also a fairly lightweight, it doesn't, it's not like a big Java framework with thousands of files and it's not like JRuby where it needs to load up a full runtime before it can start serving requests which is hard-wired performance. And it's also, so in a small footprint in the dependencies. So it's also kind of like Rails and it's sort of designed in a way so that it looks a lot like Rails so here's an example of some dubious code. So it looks a lot like a Rails controller. You've got actions and using instance variables and Mira looks a lot like Ruby because it uses very similar and it will actually the same sentence. There are some differences though. So because it's Java, you're importing packages and some of the niceties of Ruby using proxies and things are not there yet. And in particular one interesting thing is that the ERB things are made using functions, I mean methods. So how these are defined is you use macros to define the different views and so you specify the name and then the method you want to define and the location. For models, it uses a framework that looks kind of similar to how a data map works because like Bob was talking about App Engine's data storage is scheme-less so you just define the properties that you want and then you're able to assign them pretty easily. And it looks like, you know, so it looks kind of like data map work but it's a little bit different because it's a different language and it's a different part. So when you're doing a template, it's kind of the same kind of deal. So it looks similar to Rails but it's slightly different because of some of the differences between Ruby and Mira. So true scheme examples, perhaps. Good answers, yeah. So the stuff that I've been working on with this is mostly around tool chain issues so I've been working on trying to make it act a little bit more like how Rails does when you're building projects. So you can do things like doobies new and that will generate an app skeleton for you or what, depending on. So that's pretty much it, I guess. So if you wanna check out some of the code and things, there's more information on Mira.org about Mira and then if you wanna look at doobies, it's on GitHub, under Mira. There's also a couple of demo apps on appspot.com. So Rails Analytics is an example of an application that has both Rails and doobies code running in the same instance. So that's one of the upgrade path things is you can write, you can take a Rails application and you can take the controllers and models and translate them directly into Mira code and leave them more or less in the same places in the application source structure, which is, you know, so it's has a nice mapping to that. You can also check out the Mira mailing list. Doobies doesn't have its own mailing list right now because it's kind of small. All right, so I'm gonna be talking a bit more about Google APIs. My primary responsibilities are actually on social web stuff, in particular Buzz. And for the Buzz APIs, and actually some of other ones, latitude, moderator, and some upcoming APIs, we've been identifying some problems in the way that we've been designing our APIs. In particular, we've been identifying maintenance issues with how we're writing clients. So I'm gonna go over some of the things, the lessons that we've learned and how we are correcting our past mistakes. And also I'm gonna go over some cool stuff that we have built as a result of these lessons. So first, we've got some really cool Google APIs. We've got some new Ruby libraries that we've built. There's a new Ruby API client and a command line tool. So first the problem is that we've got a lot of APIs. And when I say a lot, I mean a lot. And we've also got a lot of languages that we have to support. Typically we'll support about five languages per API. So when you're dealing with literally hundreds of APIs and five languages per API, that turns into quite a lot of clients. And a lot of those get out of date really quickly. Not a great language gets the same amount of love. In the past, Ruby has not gotten as much love as we would have liked to give it. So we're trying to resolve that issue. And the way that we're going to try to resolve this is with a new generation of discovery-based APIs. So the idea is that instead of writing lots and lots of clients, what we're going to do is we're gonna write one client. We're gonna have one Ruby client for all of our new APIs. And that one Ruby client will have basically pluggable parses. And potentially we can even share formats across APIs. So that way we can even have a single parser that will serve multiple APIs. So for example, one format that we're supporting very heavily is activity streams. So we're building parsers for activity streams that will potentially get used across multiple APIs. And because they can just plug into this one client, our maintenance costs are much lower and we can do a lot more really cool stuff to make the Ruby clients more mature and the parsers more mature. So it's a big win for the Ruby community and it's a big win for you guys. The formats that we're using, I mean there's been a lot of work in the past in this area. I mean obviously it was still not the greatest thing in the world. And also there's Waddlef, which is a much more modern approach. But both of these are basically XML formats, which means that you have the overhead of all the parsing you need to do and whatever additional steps you have to take there. Our approach is actually, looks a bit more like this, it's built on JSON. It's actually, I mean it's a big block of text and it's kind of hard to see on the screen, but it's actually fairly simple and easy to parse. Simple enough that you can actually do it on a mobile client. It's very similar to Waddlef, but as a JSON serialization. Like I said, it's lightweight, it's easy to parse. There's a much lower overhead, which is extremely important when you're trying to deal with a mobile client like the iPhone or Android. And another interesting property is you can essentially pre-generate wrappers that will allow you to bypass much of the overhead and the round trip of actually heading to discover the endpoints. And that's extremely valuable on mobile as well. For Ruby, in the process of building this client, we discovered that there were a lot of deficiencies, especially on the authentication side. The OAuth gem that currently exists for Ruby is, sorry to the author, but not that great. It had a lot of design flaws that were going to be a serious problem with this type of a client. In particular, we wanted to enable the ability to essentially swap out the HTTP client that you use underneath. So the idea is that you should be able to, if you're using a vent machine, you should be able to swap in a client that will work with a vent machine, and that should just happen seamlessly. If Net HTTP has been enough for you, then you'd be able to use Net HTTP. Additionally, in order to enable this, we had to build an HTTP client abstraction as well. And so that's also available as a gem. And then finally, we have the actual client itself in the command line tool that works with it. So Signit is our new OAuth library. It currently supports OAuth 1.0 RFC 5849. We're working very hard on OAuth 2.0, and when I say it's coming very, very soon, it's actually like my hacking project for the conference. So ideally, we'd like to have it done by the end of the conference. It's really clean. The interfaces are very tight, well-defined, well-documented. We have an extremely thorough test suite for making sure that we really test this thing to make sure that we aren't having like regressions or any crazy stuff going on. The HTTP adapter is really, really simple. I mean, it's like three methods. And it essentially uses the rack tuple spec for representing responses, and then a quad tuple for requests, which is very similar to format. Adding new support for new clients is really, really easy. It's like one new class and done. And then the Google ADI plan is a preview release. We aren't quite recommending it for production use quite yet, but it's getting there. And if you want to install it, that's gem install Google ADI plan. It's open source. The official home page is on code.google.com, but there's essentially a mirror around GitHub as well. Shh, don't tell him when to Google. I did that. The Google API command line tool is actually really cool too. This is the help screen for it. It's got a lot of capabilities. It can do some really neat things. When you're debugging an API, when you're just getting started, you want to try out some cool new stuff. Usually the first thing you want to do is break out of curl and go, okay, so what does this thing actually do? But if you have an authenticated API, that can be really difficult, because the curl is just going to give you a 401 response because you didn't pass in the appropriate credentials. And so in the past, we've actually used a tool that we call OA Curl. OA Curl is a Java app that basically just runs on the ground line and lets you pass stuff in after authenticated as a particular user. However, it's kind of error prone. It's really fiddly. So we built this instead. It has a couple of functions. You can start off by logging in as a particular user. It also has the ability to access the discovery endpoints. So for example, you can list all the methods available on a particular service. So like in my demo, I'm going to show basically how to discover everything that the Buzz API can do. It can then execute a method, pass in whatever parameters you need. And then when you go to write the code, sometimes you just don't really care about the authentication piece. You just want to hack on some code. So we also have an IRB command on the command line tool that basically just boots you up and gives you a reference to a client object that's already had the authentication piece all set up for you. So I'm going to jump into a real quick demo for it. They can fit on that screen. All right, so the first thing we're going to do is we're going to log in with the Buzz OAuth scope. That's just a simple command Google API OAuth log in and you pass in the scope that you need. And this will open up a web browser. And I just appreciate that, here we go. We'll open up a web browser. You can see that it has already run to the appropriate page. So I will sign in, if I type in the root password. All right, so you get the access screen for good paying. Guess what, we're in access. All right, so now we're back here and I'm authenticated. And now I can do cool stuff like actually API calls. So first we'll list all of the posts that I've created. So you get back with JSON's blog containing basically everything I've ever posted. And you can see that it's actually personalized. There's some, yeah, so you have Google Buzz, Self-Feed for bottom. So it's actually authenticated as me. And then let's create an actual post here. So I am going to run this command. And I've got a post.json for our already set up. Let me just cat it real quick so you can see. So it's basically just a little json post. You should get a response back. That basically contains the response from the API. And now if I open up, you should be able to see. So just create it. So you get a really, really clean, easy way to make API calls that are authenticated. And that's the command line tool. So one last thing I got to show before we run out of time here is the IRB. So here we have a client object. And it's actually already set up for us. We're already signed in or authenticated. So you can just say client.execute. I want to get, actually I have it already in my buffer source tab. So I activate that list. Pass in the parameters that it needs. I want json pretty printed. And you will, so I have to, oh, insert. That's an insert I need for this. So there's your content. And it's the same content that you saw me found. So anyway, that's all we have time for. And I'm gonna hit our last slide so you can see where to take a look at this stuff. You can fork the code on GitHub for any of those three projects. And these are the addresses where you can find stuff. If you just go to GitHub, start monitoring the top projects. Thank you.