 Kind of a broad topic, integrating web applications. Wanted to kind of talk about Ruby and web and how things go in architecture and stuff. And so that's what we'll get started with today. So I want to start with kind of how we got here. Big picture for me anyway. For me, Ruby in 2005 was Rails. And Rails in 2005 for me was how to write a blog in 10 minutes. How to build another to-do app. A lot of just kind of simple models. We were kind of celebrating the, he's distracting me. So you know, kind of celebrating the ORM and building web apps that have just a sprinkle of Ajax in them. They were pretty simple back then at least for me. And Rails was a lot of just about trying to get our code organized back then. 2010, last night I did just a Google search, looking for tutorials on Rails 3. Some of the things that were coming up were bundler, active relation, active model. You know, bundler is because there's so much going on in the Ruby community. Dependencies and issues and things. It's complex. We've got to kind of simplify it with bundler. Active model to kind of modularize things, active relation. Some really neat things that are more about integration, more about big, more about there's more going on. Our applications are more complex than they used to be. At our company, Fleet Ventures, we're building a video rendering application in the browser with Rails. We're building a system that works through the patent application process. This major document parsing and machine learning on that. We do lots of medical records, coding in the browser for medical records. Just kind of complex stuff that we do now. So we come a long ways from just simple apps. And the problem I'm having with that sometimes is I make, or I start with big monolithic apps. Tough to, you know, big, big Rails apps, big, big apps that have lots of models and lots of gems and lots of dependencies. And in 2008, Justin Getland spoke at RailsConf. It was about something almost similar. And his talk was called Small Things Loosely Joined, Written Fast. And the concept was if we architect things smarter, we get more done with less resources, we can maintain it better. It's better code. And that's, I think, kind of the trick with integrating web apps. And going smaller, going simpler, using other services, don't try to be everything. Solve one problem in one context and go on and do a different problem in a different context. And there's a lot of ways to do that. And so, you know, unfortunately, you know, like I was saying, I've learned the hard way sometimes with these big apps. And so coming back and refactoring things and learning to use web services. And so there's kind of three basic areas, I think, that make sense. Internal systems, you know, we're building internal web systems, external systems and open systems. So I'll kind of break it out that way. So internal systems are basically, you control what you're doing. Maybe you have a big monolithic app. Maybe you just think that you're gonna reuse some code or maybe it's just easier to do something smaller. At the RailsConf this year, there was a company that had built a learning management solution and their talk was entirely, just saw the slides, but they'd used 30 Rails apps to build this. They just kept it really modular internally and they were able to keep things kind of the way they liked to. So internally, if you control everything, you can build your applications in a simple way and you can just have a simple decoupled system. Maybe your search, maybe five or six models all bundled up that make sense together, that you build smaller apps. It's cool in the Ruby world to use RESTful. RESTful connections that's been around for a few years. Basically, a typical Rails controller like you guys all know that allow the URL to kind of tell you what you're after. So you can create, read, update and delete your basic models through REST commands. I'll go in that just a little bit. And I think that if you're gonna go from scratch and it's not already a Rails app or a Merv app or something, stick with Sinatra. It's really easy, I'll show you how easy. And there's a lot of ways to consume it. REST client's kind of the easiest way as well. So here's a Sinatra app. Require a bunch of libraries more than you typically do, but I did because I did. And basically you have to require Ruby Jam to require Sinatra, that's what you need. The rest of this, I was using Data Mapper in this and I used Bundler to keep it clean. But basically you need to require Sinatra and then I required some local libraries. I set up the database, but the bottom line there, the post documents is just one method. It's a RESTful interface for creating a document and it just creates the document in the database. And so that's one page, it's a web service. It's local, it's easy, it's simple, it's a haiku. And with that you can go and store and save a document. And if you put another 20 or 30 lines you could view them and download them and update them and all kinds of things or do related things. And you can imagine if you had 20 or 30 little methods in there for anything you wanted to do to a document, pretty soon you've got a web service that does just documents, uploads and handles documents. And now you've got probably a good start to an internal web service that's gonna do a good job. And we actually kinda started this last night, we're gonna get to this in about two weeks here at work where this will actually grow into a real web service that we're using where we're gonna do a lot of behind the scenes work on these documents in a message queue system and just having a simple Sinatra web app to get it in and out and get things started is all we needed. So that's what that is and then to consume it it's even easier. The way I did it is just was from the command line. But REST Client, if you just suit or gem install or gem install REST Client you'll get a lot of tools and it does a lot of smart things for you and that's kinda why I like REST Client. Here I just said REST Client, I gave it kinda a base URL and I was running the Sinatra app locally and so I called that method, that post documents and I named it and actually sent a file, I opened a file and REST Client smart enough to know that that's a multi-part submission. Yes. Is this a gem or is it? It's a gem, yes, it is. Yes, so it's a binary that comes with a gem. There's usually you'd use it like in one of your other apps or in a class or something and you can just do capital R, REST, capital C Client. And it's really easy documentation. The reason I like it is because it's easy. But right there it handled multi-part uploading, it loaded it and created a document in my database and it's just that easy. And so if you control the consuming and the producing if you use REST Client plus Sinatra that's probably the easiest way. With Rails there's a little bit of you know things you're thinking about protect from forgery, some configurations you're thinking about, authentication. So you're a little more involved. Sometimes if you already have the Rails app though it's worth it. You already usually have it in the right format ready to consume and produce. So internal systems breaking up your systems may be a thing to do. Definitely if a product's getting too complicated. There we go. External systems are everything else. All the web 2.0 stuff that we all consume. And we've probably all consumed them with our Ruby apps or they're not hard to do. The basic way you usually start is an adapter and API. If you have to you might custom build something and here's the cool stuff. I'm starting to realize that Datamapper can be really, really powerful for web services. So wrappers and APIs, there's kind of for everything something already usually. I just checked this morning on GitHub. I was looking actually at the results and they all looked like they were wrappers or a lot of these were 1831 wrappers or so in Ruby on GitHub today. And pretty much anything and everything you want to do. All the web docs, all the web applications that you might want to consume and use and bring them in. And they're usually really basic. You look at the documentation, there's usually a class, there's usually a key, it's usually an hour or two to get a web service integrated to what you're doing. And so that's not too bad. Every once in a while you'll find something you just don't like, the wrappers, you just don't like the APIs, you don't have one. So you have to custom build it. Ruby's got some good ground work that we can start with with that. XMLRPC and SOAP are both part of the standard library and so you can do different styles of consuming services that way. If you're doing really old school stuff, EDI stuff, they have these standards, these documents that are these piped limited documents that they come up with 20 years ago or so. And they run a lot of industry, a lot of the medical world that we work with uses X12 to talk between different entities. And so there's an X12 parser that looks pretty good, Ruby, Ruby Jam. For several years now, Active Resource has allowed us to consume RESTful resources if they're out there and there's not a good wrapper or anything there, you can bring it into your Rails app or you're using Active Record. An Active Model is a way that it's not really about web services, but it's kind of about making your plain Jane Ruby classes kind of look almost like an Active Record, validations and callbacks and some of those state management things you can kind of add. So that's a Rails 3 thing with Active Model. So there's not a lot I can say about how to do that except if you just do it. But something that I was realizing as I was doing it is a way to simplify it, at least for me for things is the Data Mapper adapters are incredibly powerful. Data Mapper is really, really modular and they did a great job of extracting out the adapter. And if you've got an adapter, you've got all your finding, all of your associating, all your validating, all of your everything is there. And to create an adapter, you have to write four methods, create, read, update, and delete. And they give you, if the couple lines of code, you would have 41 specs already running against your adapter. So basically you set up the specs, you've got 41 things to get right in four methods. Once you have that, you have an adapter. So what we're looking at right now is an adapter or a DM Salesforce that was written for Salesforce. So Salesforce is a web service, kind of an ERP type solution for customer relationship management. And they developed what looks like to a lot of you guys. That's just a class, like a Data Mapper regular class. It's, they set up the repository to use the Salesforce. So you can have a backend database in your app and you can have a Salesforce backend here and here's an account that's, they define just the properties they were interested in. And then over here, they have a contact and they connect the contact to the account and then they consume it and use it just like it was, just like it was in the database. So this is a web service that's running like it was just part of your app. So it's like you wrote Salesforce. So really powerful. So look kind of an art to that. But I think really powerful way to approach web services. So that's kind of the external things. Now, real quick before I jump into open resources, kind of the art of it as I've used web services, there's sometimes latency issues. The way I usually get around that is I ask for something asynchronously either in the backend or I get the webpage up if I'm needing to view it. And then I ask in the background to get it and I just have a JavaScript listener to tell the user when we're ready to go. So that we can kind of fudge a little bit on how long it may or may not take to get data from a web service. Cause maybe it's slow that day or maybe it's down or whatever. So I kind of decouple that and that's important, I think. Also other things that you deal with when you're doing external web services is some of the authentication security issues. I haven't found any general rules of thumb for that just every time I work on it. I'm thinking a little bit more about that than maybe if I'm just building my own little app. But open resources is another web service that works today. This is kind of a web 1.0 and a web 3.0 examples here for open resources. Sometimes there just isn't an API. Sometimes there just isn't a way to wrap anything. There's just data on an HTML page. And so you gotta scrape it. I haven't done any scraping for about a year but I had quite a big project. I had a client who needed their data from their own supplier and they couldn't provide it in a useful way. So we ended up scraping the supplier's site. And it was important, we got it right. And so I ended up using a mechanized and HPCOT to get things done. The structure I ended up doing was I tried to treat all the pieces of the external website like classes and methods and try to group it up. So I used OpenStruct a lot and so here's an example I just try to basically bring it down to the level almost like it's a local method call. And that way I can run tests. And I thought, it turned out it was important that I wrote tests because things change out there when you're scraping websites. And so having tests around tell me whether or not my scraper still work is important. So this is one where I just set up a few variables and then I just call something in the class to start asserting data that should be sitting in the document that I just created. So I tried to just treat it like regular, like it was a class in my library, like it was local. And that was I guess the biggest thing I learned from that experience. But the issue we have even web 10120 API or not is there's a lot out there, it's exponential and it's growing, it's growing exponentially. And you can't keep up. If you're thinking about it, you've got a web service out there and every time there's a web service they've got to develop an API and then somebody's got to write a wrapper and then you've got to adapt to that wrapper. There's a lot of work every time you want to use something. And you don't usually tie into a whole lot of things but there is another way. Turns out, and this isn't for me, I was listening to a guy in London talk about linked data and turns out that Tim Berners-Lee a few months before he even coined the phrase World Wide Web, he was talking about linked data and talking about Web 3.0 stuff. And kind of the idea that a resource or a link on online should actually relate to something concrete and be semantically discernible is kind of what I'm saying in a funny way. But maybe a simpler way to say it is a linked data, some suggestions to make linked data work for you is name things with URIs. And go ahead and make those URIs point to real HTTP pages, something you could actually go consume and look at. So if it's new, you created it, go ahead and publish that, put it on a web service somewhere. So, and when you do that, put something useful there, something about what you're talking about. So if we're talking about this web conference, we might just make up our own link, a URI that might refer to it that has some RDF behind it or whatever and just make something useful or just point to convert maybe the main conference website a little bit to have linked data on there. And then finally, link to other related information so people can traverse it. And it's really interesting because if you stay simple with linked data, it gets really, really powerful because now your database is the web, whole web, everything that's using RDF and linked data and these ontologies and the kind of infrastructure that we've been building up for quite a few years. And so now web services don't have to mean an API, don't have to mean something that you spent time building a wrapper for or learning the API for or anything like that. It can just be something you can go out there and discover. One of the most powerful things I ever saw with this stuff was six lines of code application. It asked, what is the relationship between a fox and a cow? And the answer got back was they both have commercially viable pelts or coats. First, skins, that's whatever it is. And it figured it out. And it was what they did was they asked the USDA, USDA had published this huge RDF database with all this information and it just looked for what's the most relevant way these things link. And so it's really, really powerful if you start to explore and look around at things. Wikipedia, a lot of Wikipedia has been converted to another system called DBpedia, which is all the RDF information for the same thing. If they can extract links out and meaning out of the things you're looking at at Wikipedia, it's there. The BBC has got major commitments to that. Other huge, huge resources that have amazing amounts of data you can consume and information you can consume, you can relate to your own web services. In the Ruby world, turns out we have some pretty good ways of doing that as well. Geminsol, RDF, RDF Raptor, RDF JSON, RDF Tricks and Spira. RDF Raptor is sometimes a bit involved. There's a good link for getting through all this. There's a really good blog that's keeping this RDF RB stuff all intact. They're not very active writing, but when they write, they write really well about this stuff. And then so if you install all that stuff and then you just maybe start an IRB console up and require your RDF, RDF and triples maybe in Spira, just a little simple application here. I added a graph from this external resource. I consumed a web service and for everything in there, I just put the note that was out there and a bunch of gobbledygook, but it's real in there. It's information about Jay Hacker, his name and a lot of things about him. And that's okay, that's useful. Spira's a little bit more exciting. It feels more like a model and it's a little newer. Create your repo, same repo, that's what we had in the last example. You add the repository to Spira and then you go ahead and create a simple model. And on this model, I just said, hey, there's two properties I've interested in. There's about 10 on it. What it's doing is conforming to an ontology called FOF, friend of a friend. And so it already tells you, we already know that it's gonna have name and nick in there and it's gonna have other things in there too, account and things like that. But we just said, well, all I wanna know since I'm gonna be using these people is their name and their nickname, but you could set more things up. And you see over here, the predicate FOF name, it's basically saying I can go look through this data and figure out what we're looking at. And I go ahead and I instantiate Jay Hacker to some URI that's out there and I say it's as a person or as this model that we have there. And then we go grab the name and the nickname. We can save it, we can change it. We can treat it like a model. So there's no API that we had to learn extra. There was no extra service we had to run through. There was no new connection we had to deal with. And so it's a lot better if it's available to consume web services and kind of what they're calling the Web 3.0 way. So that's the basic ideas I had for web services and integrating web apps, kind of a broad topic. Kind of fun and embarrassing. Last night as I checked my blog, it's down. So fleeventures.com, we'll figure that out and debug that today. But I've had about 50 or 60 links on the blog there that tied everything I talked to. Probably take you two, 300 hours to find all that information of the same stuff. So if you wanted to kind of think about that, there's some good stuff to work with. So thank you very much.