 So I thought it might be interesting to talk about briefly about strategies to evaluate open source projects in an enterprise environment when in the context of evaluating software that a task at a company or on behalf of a client, where strategies to evaluate proprietary software are somewhat well developed, people have been doing it for a long time and have developed strategies that come up in formal evaluations and come up sometimes when the government department puts out an RFP, an RFI, some kind of tender process and they say well you have to answer all these criteria. And some of those criteria are a bit hard to apply to open source projects, but we still want to use open source projects and we still want to engage and be part of what the open source project community is inspired and being within the context and the problematics of being in an enterprise environment need not be a problem for that. But there are a couple of things that say there are new strategies that need to be used and explored. I'm pretty sure that a good part of you have already used these strategies consciously or unconsciously, but I have been through a couple of such exercises and found myself forced to make them explicit and write them down and write them as a strategy. I have some notes here that I was writing, but I don't have them complete, but the strategy we have used and we have been very happy with it has been to go through the initial evaluation that you do with any other software you want to make sure that the software you're looking at is fit for purpose, that it meets your stated requirements when you start the process. And then you use those early requirements to build a short list and to narrow down your scope to two, three, four projects that you're really going to evaluate in depth. And at the in-depth evaluation stage, there are some new criteria that enter the stage because we're going to have the source code and that's a new thing because with a proprietary application, perhaps you get the source code, perhaps you don't, but you install it, you run it, you use it. But having the open source community around the project available and being an important part, and it's going to be an important part of the success of your internal project or of your project for your client. So I looked at how proprietary projects are evaluated and you will see in tender documents that they say show us whether the company that supplies the software has been profitable for at least five quarters or six quarters or eight quarters and they say what is the performance of this company, will this company be viable in three years? Because whenever you're evaluating an important piece of software for your infrastructure, you don't want to be left without an upstream, be it proprietary or not. So the kind of things that we started looking at was asking ourselves what is the number of uses that the software has and what are the motivation and the environment? What are the level of interest in this software? So we went out and took measures of mailing list traffic. We took measure of that mailing list traffic historically, trying to see how long has the project been in use? How many people are using it? Is it growing? Is it dwindling? Is it are there spikes around releases? What releases have been popular? What releases have problems? And then we also looked at the spirit of the mailing list and are people flaming each other? Are people helping each other? What is the spirit of this community as a user community? We also looked with very similar criteria at the developer mailing lists or forums. How long have they been running? How open is the development? And in particular there we found that there are many open source projects that are open source because they carry an open source initiative compliant or trademarked license. And that is very common. You have a personal need. You have a personal itch to scratch. You have a company need to scratch or a group or community need to scratch. And you say, well, I'm going to find a free open source software solution to this. And you go out looking for it and perhaps you Google around or look at Freshmeat. And there are heaps of piles of software that come up as open source. And when you scratch the surface a little bit, they are perhaps in-house softwares from small or large companies or from universities or from different institutions that were not selling it under a license before, but it was an in-house project, thousands and millions of in-house projects today happening, programming projects. And they said, well, we have this in the corner closet. Why don't we put an open source license on it and put it out there? Hey, it's in Vogue. It's cool. It's what you've got to be doing. But that doesn't bring a community with it. It doesn't bring the open source that we expect as the whole package because the whole package has a lot more than software. A lot more than a license statement. So clearly, when I'm looking at a tool that I'm going to be using for the next three, four, five, 10 years, I want an open source project, not a software with just an open source license. So we were looking for groups of developers that belong to different companies or were independent or were purely volunteer or a mix of both, or we were looking for enough interest from people with a diversity of motivation and backgrounds that no one had enough control over the project to take it away from the community or to fork it in a significant way that would leave the rest of the project in an unclear state or in an orphaned state. Well, that's where my notes end. But we looked at a lot more things. One of the most important things we did and that I initially did experimentally and as a little bit of an experiment and it worked out so well that I am doing more and more is when evaluating an open source project, I set out to fix a bug and implement a feature and make sure that both things get integrated upstream. And a degree of my success is one of the most important measures of my success with the project or of viability of the project is if I find an obvious bug, how long does it take me to understand the code enough to fix it and what does it take me to make sure that it becomes part of the next release? How positive, how productive is my interaction with my upstream at that stage? Similar for some trivial feature that I may find and I'll implement just to go through the exercise of working with my upstream because that's what I'm going to be doing for the next two, three, four, five years, ten years. And that really shows that has really shown to be a very good measure of how projects are going to evolve. Every time I had a patch rejected or without a helpful rejection, I don't want to see that patch, I'm not interested, someone dropping the patch or saying, I'll look into it and I had a couple of security, interesting security patches, we'll look into it and never got any reply. Those projects stagnated, a year down the track, the project is still in the same place. The projects where there was a positive interaction, perhaps someone said, oh, you had the problems there, but the patch, I don't like the patch, perhaps the patch should be reworked, I'll take it back, rework the patch, send the patch. And where the dynamic worked, one year later the project is growing and growing and more, more developers, volunteers, privately funded, all sorts of interests are playing, joining the game and moving the project ahead. So this exercise, which by nature of the way Debian works, we are all engaged in, it's also a very reliable indicator of projects having the required elements for success as open source projects in the long term. And whenever you're picking your tools, you really want to pick successful tools because changing tools in two years' time, because key tools for your infrastructure, changing key tools for your infrastructure in two years' time because the project has stagnated because it doesn't have the right social components or the right technical parts, the right technical architectural decisions in place, it's a major risk that you have to manage and you don't want to expose yourself to it. Then working with the project, we have, as I mentioned earlier, we have used ARCH in a mix of TLA and BAS which, depending on the release versions right now, interchangeable sometimes with good luck and sometimes BAS works a bit better in some cases and TLA works a bit better in some other functions, luckily they read each other repositories for now. And we've been doing a lot of branching and merging and it has been really useful to use patch-based version control systems, although, as I'm mentioning, it's important also to be able to support convergence. You want to make sure that your SCM will tell you when you are fully merged or what parts you're fully merged and so that the merging story, the merge back story yields a reasonably clean slate for your next cycle. And we have found it very useful to run our branches publicly, so they would be in a mirror and available for the upstream project to see because we would talk about bugs that we found in our repository and we fixed and we're back porting to the CVS, so we've used Arch actually to run a vendor branch style. What in CVS you would use as vendor branch, we used Arch to track that and we merged upstream, we merged into CVS and it's extremely painful, exceedingly problematic. That's one of the reasons why CVS is a real bottleneck for our current open source project infrastructure because it's really hard to run a private branch for a while, develop a few features, get them tested, get them sorted out and merge it back for the next public release, for the next open source project release. But we've been using Arch to keep track of our internal patches, with a lot of patches, we pushed back 90% of them and it's been working really well, and it's been working really well with those projects where the upstream was passed that initial evaluation with flying colors where it was easy to work with the upstream, has proven easier to work with the upstream and with the code, in project and keep ourselves from diverging too much. With those projects where the upstream has been a little bit harder to work with, we have ended up diverging more because some bug fixes never make it upstream, we keep growing in our divergence and it's harder and harder to manage and then you start running through the tensions of forking and running a private fork is really, really not a happy situation, it's not what we want to do, it's not what anyone wants to do because it burdens you with a lot of development and you're going to stagnate the snow other path than to merge back to keep a brace of the software that is being developed. All of this has happened not actually in a corporate environment, these projects have actually happened in a push to move New Zealand tertiary sense calls onto open source technologies, but we did an evaluation based on a corporate style framework and these are the tools and strategies we've used and we're extremely happy with it and the more for the private interest the more our approach embraced the open source philosophy and ethos, the more successfully we were, the more successful we were and the more we kept code private not by our own desire but because we couldn't collaborate with the upstream the more costly and awkward the situation was. It's a risk, the question is whether I have any good tips on how to persuade your bosses or your clients I guess and as to why working with open source is a good idea. I would say, well you can talk about how important all the internet infrastructure that runs on open source, all the base things, all your infrastructure, today your technological infrastructure runs a lot on open source. But one of the things that I found very, very strong is risk management. If you read closely, actually read the license, pull out the license of the proprietary product you're about to buy and get a lawyer and ask what the risks are and that's extremely powerful because the license says we're going to yank this from you whenever you blink in a direction we don't like and for any piece of critical infrastructure that is not an acceptable risk and if someone says ah but they're not going to do it you can say well you know there was this company called Santa Cruz operation right and because everybody will say ah you know they have an interest in the market they will keep having clients and you say well you can point at all the corporations suing their own customers and that's a very, very simple thing to point out and it's a practice in risk management that today it makes it a very, very risky move to have your key infrastructure in something that has a single vendor, single player that gets angry one day and says you know I'll pick up my toys and go and play somewhere else and you're not, you don't own your own infrastructure and it's very important for this management. Okay, five minutes late. Sorry. Okay, thanks a lot. Good one.