 Now, Kuna Rolf will give us a talk about how to bring together Deviant and Rails framework. Kuna, thank you. Thanks. Okay. Well, I started playing with Rails maybe two years ago and, well, I find it to be a very nice application framework for web development and it's becoming very popular. Yes. But I have suffered trying to integrate it into the Deviant, both for using it as a developer but even more in order to package Rails applications. We will see several of the reasons and then, well, I expect interaction. First, and here I will also request Lucas to step up a bit because he was planning to give a buff on this topic. Ruby libraries or modules are called and distributed with a strange package management system called Gems. Gems, basically, well, it's not only a packaging system but it's a way of calling modules or calling libraries from within Ruby. So, a program can specify easily which library, which version it's going to use. Yes, instead of using require this file, you say require gem and the name of a gem and when it works, it works. Yes. But, well, one of the main points is that it encourages, it allows people to use the different versions and having installed different versions of the same software. So, there are several problems with Ruby Gems. This is still mine? Okay, so, give you... No, she's at the mixer. That's fine. So, yeah, Ruby Gems has a way for users to install libraries which are not supported by Deviant currently and are not packaged inside Deviant. So, you can see it as some kind of tar, basically. It doesn't do much more than tar. One problem with Ruby Gems is that it's so intrusive. When you, for example, if you compare it with C-Pan, C-Pan is really great. Ruby Gems is not so great to say more because in the source that uses Ruby Gems, you have to use some special instructions to use Ruby Gems. So, if you want to use the same source without Ruby Gems, you have to patch the source. So, for example, you use require gem to load libraries using Ruby Gems. You don't just use require. So, if you want to package it without using Ruby Gems, you have to drop all those instructions. Another problem with Ruby Gems is that if you use require gem to load a library, and you already have that library installed but not as a gem, for example, because you installed it from a Deviant package, it doesn't work. It doesn't see that it already has that library. You end up with two copies of the same library, one from the Deviant package and one from the gem. Another problem is that Ruby Gems is very popular inside the Ruby community, especially inside the sub-part of the Ruby community with web applications. And there are lots of libraries now that are only distributed as Ruby Gems. Some parts of the Ruby community totally stopped distributing libraries as starboards because they think that it's useless. And when you talk to them, it's really hard to get them to understand why it's still important for some people to be able to get this table. So, the gemification of complex Ruby applications is very complex. So, for example, if you want to package Rails, you have to patch a lot of files, another complex application like Rails, you have to patch a lot of files just to remove the calls to Ruby Gems. So, it's not using Ruby Gems. It's very simple but very intrusive and takes a lot of time just to replace every call to require gem to just require and it should always work that well. Another problem with Ruby Gems is apparently it was developed by people who probably only were familiar with Windows. So, they didn't fix the problems that were fixed on Unix a long time ago with library versioning, with SO names and stuff like that. So, basically, you just depend on the version of the gem that you want and it gets installed. You don't depend, there's no notion of stable API for Ruby Gems. You don't version the API, you version the library. So, usually, when you start using Ruby Gems, after a while, you get lots of different versions of the same libraries installed and then you get into the same problem as on Windows when you have one application that wants to use one version of one DLN and the other version of the same DLN. So, you get that with Ruby Gems. That's quite a backward move, very backward mode. Thanks. And this is quite pervasive in the Ruby developer community. They will very often defend this, what happened here? Ah, the projector. Yeah. Okay, okay. I'll just continue talking then. The thing is, they take pride on all these problems because they are very keen on the agile methodologies. So, they say it's the right way to program. Yeah, but well, I'll get to the cultural differences in a bit. Then, well, what Lucas presented is about gems, which is the general Ruby library format. But then, if we continue getting to Rails-specific things, we get something even uglier, although much more comfortable sometimes, called plugins. Yes, gems are general modules that should be usable at any point of the application, but the plugins are specific to Rails behavior. For example, you can code a plugin that modifies the way the controllers are to be used. So, they integrate into Rails for workflow. Usually, you don't even have to call them to include them because they're sole presence. There's a file called init.rb. It modifies the Rails classes themselves. So, they just have to be in their place. And I'll get to this later. You can have a lot of things inside the Rails root directory like vendors slash plugins or whatever there. Vendor is the place for storing any code you didn't write yourself. Some plugins have been started to migrate to become gems. This means they start cleaning up some of their intrusiveness into the whole process and allowed to be called from the outside. We still need to specify sometimes some... to allow them to be included in some core libraries. The role of init.rb. So, it's a bit more of a burden to the programmer, but I hope that people will start realizing that instead of bare plugins, they should be writing gems or, well, proper modules. But, well, at least there is something sane about the gems. Gems need a version number. It's not legal for a gem not to have a version as for a Debian package. But as plugins are just dropped inside your directory, very often the... and very common plugins don't have the concept of versioning. So, we are left only with the possibility of packaging Git snapshots or something like that. We cannot also easily automate the tracking of new upstream releases because the only thing we can do is Git pull or update subversions or whatever. There's no upstream version. Okay, the vendorization. What does this mean? I think this is the gravest scene in the Rails packaging problem. Both the Rails API, the core framework, and all of their plugins are moving at a large speed. Again, they call this agile practices in programming. So, when I talk to my Rails happy friends, they say, yes, I always develop with the cutting edge, and then when I'm going to freeze for production, well, I just leave the things as they were working. So, say if I wrote an application half a year ago, I will ship it to production with Rails 2.0.2. Well, right now, the new version is 2.1, but inside the vendor directory, I will still ship the old version. Of course, this is unmaintainable. Whenever I have to fix an old system, I must remember which version of Rails it's using and it's including in its directory, and they're not forward compatible. I mean, between the version one and version two, many very popular things were dropped because we're moving and we don't want to be stuck. So, another problem here is that we provide as a distribution updates and fixes. What happens there is that they just copy over the files all over. So, if, say, I am an ISP and they are hosting tens or hundreds of Rails systems, I will probably have to enter each one of them and patch it whenever something new version comes along. And, of course, a fix will not come to an old version. A security fix will only be applied to the head of the development. Maybe, as Rails is large already, maybe to the latest stable release, but not to all the releases. So, there is a trust problem in there. Well, what we are doing currently, I'm sad that I didn't start writing this earlier because I really would have liked to have Adam Majors inside in this because he's the guy who's packaging Rails and knows much better than me how he's handling this. But there are some modifications. For example, when you create a Rails application, its directory structure Rails root vendor Rails points to user share Rails, which is centrally controlled in Debian. Although, however, of course, you can remove that symbolic link and unpack static frozen or whatever you call it, Rails tree in there. Well, the problem here is that if Rails continues to ship not backwards compatible changes, then I may update my live system and my old applications will stop working because they are linked the way we are used to. Okay, I went over this already. And there's another very different and very important issue which is deployment. One of the things that attracts many users to Debian is that you want to test a package, you don't have to worry too much about it, you just install it, and the usual case is it's ready for using. Sometimes, yes, I agree, for example, the talk Andrew presented earlier, web applications are well-known for being a pain in the ass. But I don't know, I install something, I don't want to be haggling with it in order to start knowing if it works for me. The thing is Rails has to set up an application server, has to set up a database connection, and has to set up something that links the application server to the web server, and the components for each of those changes from time to time. I've been writing for Rails for around two years, and when I started with it, the favorite deployment strategy was to have a Apache server using ModFCGI, FCGI or FastCGI is a protocol for handling requests to an ongoing running application for an application server, and using PEN for load balancing and maybe clustering. Then someone said, well, FCGI is quite troublesome to get installed. Yes, I know, it can take a couple of hours if you're not used to doing it. Let's use Apache plus FCGI, which is easier. Good, everything moves there. I even uploaded to the VM the Rails FCGI package, which didn't even enter on stable because I just, one week later, requested for its removal because it was no longer HIP. Yeah, it was out of fashion. Then it's any web server, because web servers are dumb, they're not supposed to be intelligent as Apache is. So any web server plus the Mongrel application server, which is what we're doing now. But right now there's another idea that's called Passenger or Mod Rails for Apache only. I don't know, I haven't even touched it. We will be shipping Lenny with Mongrel and that's it. Rails is very friendly for developing web applications, but it's a pain for deployment. Well, deployment is one of the areas I think our distribution shines. Right now there are no Rails applications yet distributed in Debian, which is somewhat strange. There are many great applications out there, and it's very popular, but yes, nobody had until two days ago taken the pain to build a proper package. Coincidentally, when I was writing these slides, somebody wrote to the IRC channel, to Debian Ruby, hey, I have this Rails application package. Can you test it and tell me why? Okay, I learned a bit from it. There's a long way to polish it, but I think we will soon be able to upload it. The thing is it's a mess inside, but well, we should get some Rails application in Debian, and I am sure that if you install a web application, you don't have to worry as a user which framework is based on what language is it. You just want to use it, so the deployment must be addressed. I don't have this yet, but well, the general developer community culture. There's a very big cultural difference between Debian and the Rails community. As Luca said, Ruby was already decently known, and it's a very nice language, but it was not popular until Rails came along. So if you see the early Rails, the early Ruby programmers, they are very similar to the Perl culture. They are very unique-minded, which is great, and they have very good quality software. But the Rails community is usually much more Windows-minded. So they have all these problems created by Gems and similar. In fact, Gems recently became the official way of packaging libraries, which is very sad. The Rails developers tend to look for the newest features for their production systems and tend to value less long-term stability and maintainability. Interaction with upstream regarding older versions can be problematic. Yes, I will not get from most developers their attention to fix a bug in an older release. I expect if there's a Rails pusher around here, he may be angry at this, but I think not committing to supporting a stable release is not thinking about the user. In the end, if we're not thinking about the user, we may just as well not publish anything because it will only be usable for ourselves. Extreme programming or agile development works very good if you are a great programmer and do it for inside work, but we take pride in doing this for everybody and doing it well. That's what I have here. As I tell you, I recently got this person who sent us an almost usable Rails application package. The thing is, for example, he has in his application directory, I'll just show you as a simple example of what's wrong there. Yeah, yeah, wait, I think I'll try to change the profile. No, to make it black over white. Ah, very good. This is the typical Rails layout, but this was really amazing to me. We see a regular Debian rules here, and here there's something quite odd. He's removing because LinkedIn complains all those licenses. Yeah? Why? Because in vendor plugins, each of those should be a separate package. Yes? So we have a source directory that's eight megabytes out of which, well, at least, I don't know, this is at least two megabytes, should be completely sent somewhere else. And I am sure that here we have a lot of extra craft. Yeah, so, well, my point here is that I am very interested in making this work. I don't know if any of you have experience either writing or working with such things, but yes, please. Please, just wait for the microphone. Oh. Hi. Okay. Yeah. How have you not given up yet? Sorry? I mean, how do you... I am impressed and amazed at your persistence, and I wonder, on a personal level, how you have not given up. Well, you know, the thing is, I am very persistent because it's very nice to write applications in Rails. That's what I do for a living. But I want my applications to be usable in the way I think a software development should work. Yeah, actually, on a more serious note, when I work on the gems issue, I always end up thinking, maybe we should just drop all of this from the behind because, actually, it's not the way... Well, it's not useful to most of our users because the way we package it, it's not that useful for them. Well, I mean, in stable release, there's no point in packaging one version of Rails. Since, anyway, they will break background compatibility with it in six months if we are lucky. So we could just... Well, I'm not sure that... Maybe we should seriously ask ourselves if we shouldn't just drop all of this stuff and don't care about it. It would be very sad because, well, after all... Yeah, I know that as a distribution, we should try to cater more about the applications rather than the development frameworks. But if we want to... I mean, if we're going to ship an application that includes Rails, and then another one, and then another one, and there are different versions of Rails, it will be also very painful. So we have two problems at hand. One is that the Rails developers don't... The people that make Rails don't really... They don't have a sane release schedule and policy. And the other is that the developers that develop things on top of Rails ship everything with their application. So have you tried to talk to upstreams about that? Some upstreams are very cooperative. For example, well, I just started packaging a couple of the gems or the libraries or whatever you want to call them. This will paginate. It's a simple pagination thing for an object relation mapper. Well, the upstream is most willing to interact with me. Great. And I think, for example, the Rails core, because they are very good developers in the end, the Rails itself also interacts properly. But the problem is there's a whole lot of modules around that we also need to work with. I am Antonio Tercero. I'm also a member of the Ruby Extras team. And my personal testimony is that I also work for living with Rails, developing Rails. And I end up packaging everything that I need to use in my applications that are not packaged yet, which is what most of the people in the group do also, I think. By the way, thanks for packaging with pagination. And sadly, which I do is just unpack the latest version of Rails that works with my application, because I'm working on applications which are rather large at the moment. So migrating for the next version of Rails requires a considerable amount of effort. So I think that what Lucas says, even if it looks some way a little sad, maybe it's the most viable way to do, because it's very tough to keep up with all those changes. Several useful applications use lots of plugins. And on the other hand, I think that it's very important to be able to package Rails applications, although I think that I didn't find so much free Rails applications worth packaging, but RedMind is surely one that just having RedMind in there is worth enough, because it's for those who doesn't know RedMind is a project management like Traki or other projects, but supports a lot of VCSs and has a lot of cool features like projects to be projects, and even for Traki non-soft, the project is very good. So I don't know, I think that we should discuss more and maybe try to find a real solution to the gems issue that can be worked out to be able to move on. Because maybe something that could be done, maybe something that could be done is to treat this as we would treat most C-based libraries, on which we have different SO names, and we can have more than one version. Not for all of the gems themselves, but for the framework. We could, right now, ship Rails 1.2, Rails 2.0, Rails 2.1. It's a burden, but I think for cases like yours where you have to support an application written over one year ago, I think it could be the way, I don't know. Actually, that's exactly what I was going to say. We have this model of development in a number of cases. We have got, look at Perl. There is each or Emacs. You have different versions of Emacs. We ship source, they're compiled into the different areas. I have used Rails mostly as just developing a web app for myself. I haven't tried packaging anything. I would like to respond to the comment made that maybe it is too hard to package. I don't think this is the case. The vendor plugins thing that we see here is an effort by people who use Rails to make a self-contained application. You can drop whatever you want, packaged by somebody else in vendor plugins, and then you can just start up your application, move it to a new machine, and everything still works. This is a solution for a platform which is not Debian. They are trying to find solutions for, you know, less capable platforms. I think we can do better. I think we can take everything in the vendor plugins thing and say this is for Rails 2.0, and then if we need to, we can support Rails 2.3 or 3.0 when the time comes. Giving up such a useful platform for creating web applications, I do not think it's the right solution for either for Debian or for people who use Debian. Well, the problem is that we're not going to modify all of Rails ourselves to implement that. So we need someone to talk with, and currently AppStream isn't really open to talking about those issues, because they consider it okay, and they think it's the right way to do it the way they do currently. So we can try, but... No, but what he suggested and what I was also saying, which is quite similar, I think would even be better regarding AppStream because we would still be providing their latest release and the releases that other people are already using for existing applications. The problem right now is that, for example, I almost cried the day that Rails 2.0 entered unstable because my development is made on an unstable machine, and suddenly I could not work anymore because Rails didn't work, so I had to freeze to Rails 1.2 on all of my previous applications. That was at least one-day worth of work. I think we have two problems here, as some people say, and one of the problems is distributing the package. So I think we should look near AppStream and try to propose a patch to Jam system. If we can fix Jam to be compatible with Debian package, I think we can have a hard work trying to separate in smaller packages that the AppStream will be able to use Jam, and Jam will be compatible with the Debian system. But I know it will be hard to Debian maintainers because they will need to work with the AppStreamers, wait for a patch to enter. I think it's hard, but I think the solution is a point or a fourth in fixing Jam system. I'm not sure if it's a good comparison compared to Jam with Python setup tools. I think it's a pain work with many versions of one library in Python, but you can do packages to Python libraries, not so easily, but not so hard as in Jam. And if it's a standard, so you have Python Central that reduces the effort of making Python library packages, maybe Jam is the point we need to work more. I'm wondering if Jam... I mean, why aren't we treating Jam just like Atari.GZ? Why does Jam have to integrate with the Debian packaging system? We can treat upstream package Jam sources, package them up for Debian using the way that we always do, and just ignore the fact that Jam exists. Yeah, no, the thing is we are doing that, but let me see if I find it easily here. Okay, so usually in Ruby, you use here require. Yeah, this is for including something, but let me see, on a random here, vendor plugins access list. One second, strange. Require Jam, yeah, not game require. Hmm, no. Well, here, let me see this. No. Sorry, no, this requires Ruby Jam, but it's not using them as Jam. Please, if you have a Ruby Open IG, it has something like that. A comment received from IRC, Gunna. Yep. As Lami says, there's been a few suggestions in the past for using Volatile for these types of web apps and libraries. Sorry? There's been a few suggestions in the past for using Volatile for these kinds of web applications and libraries. Have you got any comment on that? I prefer, I mean, it's a way out, yes, if we want to ship the latest, but the problem is not shipping the latest, the problem is that we have to ship several versions of everything. Well, I don't have an example here handy, but the problem, the inviciveness of the patches is that instead of requiring something, we require underscore gem something. And it looks in a different file hierarchy, and well, if I'm not mistaken, it can even be used to fetch the gems and install them. Please sit in the back. Hi, I was just looking online when you said the require underscore gem thing, and apparently that's deprecated now, but if they switch to using a combination of require and a separate gem command, will that make things easier, harder, or neater? Well, how long will it take for a deprecated behavior to settle? That's the thing with the Rails people. They deprecate everything. I don't know. I will have to see that. That's not to say that having different Rails versions packaged, when Rails upgraded to 2.1 and Ruby went to 1.8.7, Rails 2.0 was completely broken. So I even contacted Aidan to try to sort out the problem with the current Rails package, and he said he was already working on packaging 2.0, and in the meantime, I had to backport several patches to fix the compatibility with Ruby 1.8.7, and ended up having my own Git tree of Rails with several backported fix that I'm sure didn't fix everything. They just fixed what my application was breaking on. So I wonder if such thing happens during a freeze, if Rails 2.2 is launched tomorrow and we are in freeze and everything is broken. But there's one thing we must keep in mind. Maybe you and I are wrongly using a Debian unstable machine for our real-life development work. I mean, I want to have a reliable stable machine because I know there will be changes between my environment and the stable, but we should try to keep stable sane. There will be bumps, of course, but... What you just said about Ruby 1.8.7 breaking Rails 2.0, this won't... It's not a problem because of freeze, because Ruby 1.8.7, I mean Ruby 1.8, will be frozen before Rails anyway. So it's not a problem with Debian freeze. Okay. Well, my aim here was to tell about the state of this work we're doing. I don't think... I don't know, is there something else to add or should we call this a session? Thanks.