 Looks like it's starting, good morning everyone, thank you for coming out, I'm excited and honored to be presenting here at RubyConf. So today I'll be speaking about my experiences running the programming unit for a small government department for the last 15 years and using Ruby as the main programming language for the last 13. My name is Jeremy Evans and I have been using Ruby since 2004. I'm the maintainer of numerous Ruby libraries, the most popular of which is SQL, the database toolkit for Ruby. I also maintain a web toolkit for Ruby named Rota, as well as an authentication framework named Rodoth that builds on top of SQL and Rota. Now, I did not start out wanting to maintain these in a bunch of other Ruby libraries. So the reason I maintain these libraries is because they form the core of many personal projects, as well as the core of the applications that I'm responsible for at work. So I work for the California State Auditor and our office is in the capital of California, Sacramento, but our auditors work all over the state. And the mission of our department is to promote efficient and effective government by performing independent evaluations of other government departments. So we publish reports of our audit findings and we make recommendations to other government departments and the legislature. So our equivalent at the national level here in the United States would be the Government Accountability Office. Now all 50 states have an equivalent office to ours, but the degree of independence and the exact responsibilities vary state to state. Now before I go any further, please be advised that all opinions in this presentation are my personal opinions and not the opinions of my department. The lawyer is having at that. All right, so I now let's talk about software development as it is typically done at other departments in this state. Why I will simplify it substantially in the interest of time and exaggerate slightly, only slightly, for a comedic effect. I'm not sure how similar this is in other states or countries. I'm guessing there is some overlap. So government software development revolves around large projects. Now in many cases, there is an existing system in place. But it has various warts, such as running on a mainframe. So executives get this idea of building a modern, it's called modern system, and throwing away the old previous system. So after the government gets the idea that they want to build this modern system, before they can even start the project, they'll spend a year going through a four step project approval process with the Department of Technology. And this is not one of the times I'm exaggerating even slightly, the median time for this project approval process is over a year. So after approval, the government will start by developing a long request for proposal detailing what they think the requirements are for this system. And this request for proposal will be prepared by analysts and program managers, without the involvement of the internal developers who work on the current system. So the government will solicit proposals from companies to build the system. And after reviewing proposals and removing those that are not considered acceptable, the government will follow standard government purchasing regulations, and award the contract to build the system of the company with the lowest bid. Payment on the contract will be based on a checklist of deliverables. And there will be little consideration for how easy the software is to use, and no consideration for how easy the software will be to maintain. The contractor will build the system using their own developers, who are incentivized to check off the deliverables as quickly as possible. And the contractors will use C-Sharp if using a Microsoft stack, or Java otherwise. And near the end of the contract, the software developer will train government staff on how to maintain the system. There will be minimal if any unit or model testing done during development. There will usually be good integration or acceptance testing, but the testing will be done manually using checklists by system contractor, government staff, or an external independent validation and verification vendor. In some cases, the contractor will base their solution on an existing expensive enterprise resource planning system, such as SAP or PeopleSoft. And then the contractor will heavily customize the installation to meet the system requirements. And this will allow the contractor to claim that we're just using common off the shelf software in their request for proposal. When in reality, so much custom work is done that the benefits of using common off the shelf software do not apply. Now, there will be a large amount of oversight of this project. The project managers write quarterly progress reports and send them to the department of technology, executive management, and an external independent project oversight company. So when the project runs into problems during development, the stakeholders will be aware. However, the stakeholders are not experts in software development. And other than approving the expenditure of more money to higher additional developers or to lengthen the project schedule, there will not be much that they can do to fix the problem. I think most of us know how well adding developers to a late project actually works. As you might be able to guess, I am not a fan of this approach for software development. I'd now like to share the alternative approach for software development that we use in our department. And this is not a fair comparison, as our department is much smaller and not similar to other departments. Now first, we try as much as possible to avoid building large systems. So if stakeholders request a large system, we discuss the situation with them and try to convince them to build a smaller system initially that only does what is most important. In some cases, we are successful, in some cases, we aren't. But we always try to reduce the scope if a large system is requested. We can always add more features later after the system is working. That is what we tell people. So the idea that you should start with a working simple system before expanding the complexity, instead of trying to build this complex system from the start, has been around for decades. John Gall wrote a book called Systemantics back in 1975, which discusses how systems work and how they fail. And a couple of sentences in that book eventually became referred to as Gall's law, and Gall's law states that a complex system that works is invariably found to have evolved from a simple system that works. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. I know that Gall was not referring to software development. He was discussing systems in general, be they computer-based, mechanical, or manual. So we have Gall's law in mind when we build systems in our department. In most cases, we try to add them as subsystems or new features to our existing working systems. Now we aim to complete the initial development of almost all projects under a month with the majority being completed in about a week. And when I say completed, I mean having all features implemented with complete automated tests. Now, their systems may take longer to go into production due to delays in getting final approval from the stakeholders that requested the system. So in my experience, stakeholders are quick to approve a project, like go develop this project, and are less quick to after the system is built, putting it into production. So a typical project for us would be to take a paper form based process with one or two levels of review and integrate it into our existing internet site. So there would be a form for employees to complete, forms for the reviewers to approve or deny the request, emails for notifying the next reviewer at each step, and reports for the employee, reviewers, and management to see the status of requests. We build all of these systems internally using government staff with no use of contractors. The developer responsible for building the system will meet directly with all interested stakeholders in order to determine the system requirements. So often, stakeholders will request features that are not strictly needed, or will request a specific feature that they think will meet their needs, instead of just telling us what their needs actually are. So it's the developer's job to discuss the system requirements with the stakeholders, figure out which features are important and what the underlying needs actually are, and then try to build the simplest system that will meet the needs of the stakeholders. The developer responsible for building the system knows that they will be responsible for maintaining the system when it is in production, which gives them an incentive to design the system in a way that will make maintenance easiest. The developer also knows that if users have trouble using the system, they will be the ones to listen and respond to the user's complaints, and it will be their job to modify the system to make it easier to use. And this encourages the developer to design the system to be easy to use, so they don't have to spend their time making such modifications in the future. Now, as the title of this presentation suggests, all of our internal custom development is done in Ruby. Ruby has been our primary development language for new projects since mid-2005, and all of our existing internal custom systems were converted to Ruby by 2009. I'll talk a little bit later about the stack that we use and how it has evolved over time. Now, we do not have a large amount of oversight during our development process. The only external oversight that we receive is from external security assessments and penetration tests, which we each performed about every three years. In terms of internal oversight, the developer will work on the system until they think it is ready. And if they wanted a problem during development, they'll usually talk to me and will usually pair programs to try to fix the problem. After the developer thinks the system is complete and ready, they'll request a code review from me. So I'll go through, I'll review all the code, I'll provide a list of requested changes, and that process will repeat until all issues have been addressed. After a clean code review, the developer will notify the stakeholders who will use a development version of the system and see if it meets their needs. Now, in some cases, the stakeholders may request changes at that point, in which case, the developer will discuss the system requirements with them, agree on unnecessary changes, and that process, again, may repeat until all issues have been addressed. Now, before any of our systems go into production, we require automated testing at both the model level and the web level for all parts of the system. We perform coverage testing on a regular basis for all of our applications, and our line coverage is between 93 and 100% depending on the system. Now, since we prefer to build smaller systems before larger systems, one of the issues that we deal with is we have all these sending requests, we have to prioritize them to determine which ones to build first. So, one of the things we consider is the size of the request. We try to build smaller systems before larger systems, and hopefully that encourages the stakeholders to be more willing to accept the building of a smaller system. Another thing we consider is how often the system will be used. So, if the system automates a manual paper-based process, we will ask how many forms are submitted per month, and depending on the answer, you may give that system high priority, or even tell the requester that due to low volume, it doesn't even make sense to automate this process. We also try to consider how important the system is to the organization. So, if the current process is paper-based, we will ask, what are the consequences of losing a form? And if there are legal issues, we may want to automate a system even if it has low volume just to ensure that we can closely track progress to make sure that we are following the law. And finally, we consider who is requesting the system. As much as we like to be egalitarian, the requester is an executive, and they want us to give the system priority, the system is going to get priority. So, what have we used it before? I mentioned earlier that Ruby is the sole language that we use for software development. So, this basically boils down to the same question as what software development do we do? And as you may expect, the primary software development that we do is web applications. So, our largest application is our intranet site, called the Hub, and the majority of our development is adding new features and subsystems to the Hub. And the Hub has some pretty standard intranet features. It has a lot of information on our internal processes. It has our comprehensive manual. It has our employee directory. Each employee has a profile page showing information about the employee, such as their division, their position, audits they've worked on, audits they're currently working on, any awards they've received. The employee profile page also has a link to a map of the floor that they work on with their desk location highlighted. And this makes it easy for new staff to navigate the office and for existing staff to easily find new staff. Now, most of the new development centers around automating existing processes. So, when I started back in 2000, almost all processes were manual, paper-based processes. And now, most of the common processes are automated via online forms. So, for example, taking submitting requests to take time off, attend training conferences, get reimbursed for overtime, purchase supplies and equipment, and many others are automated. Records are kept so that employees can see the status of all of their previous requests. Now, most of the processes have custom review and approval workflows that are based on different requirements. So, some processes, the simple ones, will only require supervisory review. But in other cases, they'll be between two and five levels of review, with the number of levels dependent on the specifics of the request and the employee's position in the department. Now, another web application that we develop is our recruiting system, which is split into two parts, an externally accessible part that appears to be part of our public website, and an internal system for human resources staff. And most of the employees that we hire are entry-level auditors directly out of college or graduate school, and most of the recruiting system is designed to handle the recruiting process for these applicants. So, the recruiting system allows prospective auditors to apply to take our online exam. And after applying, our human resources staff review the application, and if they approve it, the applicant is notified that they can take our online exam. And the online exam is timed and has 75 multiple choice questions. And you have to get about 80% of the questions correct in order to rank highly enough to advanced. And the exam is actually fairly difficult, and only about 30% of applicants do rank highly enough to advance. Just assuming the applicant scores high enough and can advance further, they are notified that they can take our online writing assessment. So when the applicant begins the writing assessment, they're given a prompt and an upload form, and they have a couple of hours to write a writing sample and submit it to us. And this writing assessment is graded by our greenhouse editor, and only one-third of applicants score highly enough to advance. So after that, there's a phone interview, and then there is an in-house interview, and the system handles all the information related to those. It also handles all the internal workflows related to processing these applications, and it has extensive reporting capabilities. And there are smaller subsystems in the recruiting system that handle recruiting for more advanced auditing positions. Now another major web application that we develop is our recommendation system. So in addition to reporting our audit findings, one of our primary functions is to make recommendations to improve government. And the departments that we audit are required to respond to our recommendations on a regular basis, on their progress, implementing the recommendations, until the recommendations have been fully implemented. So the recommendation system is split into three parts. The first part is externally accessible, and it allows other government departments to submit responses to our recommendations through our website. And there's an internal part that allows our staff to add recommendations and to review the recommendations that have been submitted by the departments. So each of those responses goes through four levels of review, and after being fully reviewed, the department's response and our assessment of their response is posted on our website. And this holds other departments accountable, and their implementation of our recommendations is often considered when the legislature reviews the department's budget. We continue to follow up on these recommendations that we make to departments for up to six years after the release of our audit report. So the third part of the recommendation system is externally accessible, and it allows the legislature, the press, and the public to subscribe to be notified about new report releases and new responses to these recommendations. And the system also allows subscribing to be notified about things that are filtered to specific policy areas. Now the last major web application that we develop is our public website, which like our recommendation system is also split into three different parts. So over 99% of the content on our public website is generated by a internal system that just caches static pages. So to update the content on our public website, we just run a web crawler over our internal system, which caches all the pages, and then those web pages are just copied to our public web servers via R-Sync whenever we want to update the content. We only have about 20,000 pages in our public website, so this approach is feasible for us. So having almost all of our website consists of static pages is very helpful from a reliability standpoint. So in the rare cases where we have had problems with the dynamic sections of our public website, our users generally don't even notice because they are only accessing the static content. Now there is a small dynamic system that runs on our public website, which handles a few actions, mostly some different search features. And there's also a small internal administrative application that we use for adding reports to the website. Now as I mentioned, we use Ruby for all development. So I'd like to talk a little bit about some of the non-web applications that we develop. So for employees to log in to any of our internal web applications, we want them to use the same Windows username and password that they use to log on to their computer because we don't want to have them remember a separate username and password. From a security perspective, you don't want your web servers talking directly to your Windows domain controllers. So all web applications authenticate by using SSL to connect to a custom authentication proxy that is written in Ruby. So the web applications submit the username and the password provided by the user. Proxy first checks that the user is in a list of allowed usernames and then connects to Windows using LDAP over SSL in order to authenticate the password. We also use Ruby to implement two factor authentication for our VPN. So we use OpenVPN, which does not have native support for two factor authentication. After OpenVPN authenticates their certificate, we have OpenVPN for a statically compiled C program which uses a Unix socket to connect to a server written in Ruby. And that Ruby server then connects to Windows using LDAP over SSL to authenticate the password for the username that is specified inside the VPN certificate. And this approach is a little bit more secure than the standard approach for doing two factor authentication. So in most cases, the two factor authentication is done with the username provided by the client. So the system uses that approach for authenticating where we use the username provided by the clients. An attacker can compromise one employee's VPN certificate and a different employee's password and use the two to gain remote access. So with our approach, an attacker would have to compromise the VPN certificate and the password for the same employee. And because we have about 200 employees, that's hopefully about 200 times more difficult. And we use Ruby to download financial information from the state's mainframe on a daily basis for usage by our financial auditing team. Now this used to require a gem that could do FTP over SSL but thankfully after Ruby 2.4 was released, we were able to switch to using net FTP from the standard library. We developed custom programs to assist auditors with their audit work. So most recently we used Ruby with Capabara, Selenium, and Headless Chrome to download almost 3,000 PDFs from a separate government website that was written in ASP.net and required JavaScript to work. We then used Ruby, MuPDF, and PDFTK to extract a specific page from all these 3,000 PDFs, combine all the extracted pages into a single PDF so that the auditors could go through the results easily and include the data in their audit work papers. Now we use Ruby in conjunction with a Microsoft program named AccessEnum to produce reports on file access permissions and changes in file access permissions over time. So the former are reviewed annually with management to determine if the permissions are appropriate and the latter are reviewed monthly to make sure that no obvious security issues have been introduced. We use Ruby to check the remaining free space on our web servers on a monthly basis to see if more space needs to be allocated on any of them. And for critical file servers that are prone to using up all the space, we have a similar program written in Ruby that automatically notifies the appropriate manager whenever the free space falls below a certain threshold. We use Ruby to check that our auditors are complying with our data retention policies by scanning for material related to released audits that the staff are no longer supposed to be retaining. We have many reporting programs that use SQL to connect to internal PostgreSQL and Microsoft SQL server databases and create reports from them. So basically, Ruby is our programming toolbox and pretty much everything that we need to build we can easily build using Ruby. So how do we start using Ruby? So when I was first given the task of maintaining our websites back in 2003, they were developed as static pages using a product called NetObjectsFusion. And while I had no previous professional programming experience, I did have some exposure to PHP. So I decided to use that. In late 2004, I heard about Rails and I tried it out. And I saw it was a great improvement over the spaghetti PHP that I was writing at the time. So after a few months of using Rails and personal projects and a few more trying it out at work, I switched our internet site over to Rails in mid 2005. Now in 2008, I learned about Sinatra and I was drawn to Sinatra's much simpler approach to web development. We started using Sinatra for all of our new developments and the initial versions of our recruiting and our recommendation systems were both written in Sinatra. Now in 2014, I was exposed to the routing tree approach used by Cuba. And I saw how it addressed the complexity issues that we were experiencing in our Sinatra applications while still being much simpler than Rails. I ended up creating a fork called Rota and converting all of our applications to Rota for the release of Rota 1.0. On the database side, when I was using PHP, I wrote all the SQL by hand using Spicely Raw SQL on a database driver. So when I first started using Rails, I used ActiveRecord, which I think was a huge time saver in comparison. After being exposed to SQL in 2008, I saw the benefits of SQL's method chaining approach to building queries. I converted all of our ActiveRecord usage to SQL that year and we've been using SQL exclusively ever since. So in the lower levels of the stack, we've always used PostgreSQL database starting with version 7.1. The operating system has always been OpenBSD since we were already using OpenBSD for our firewall and we had experience with it. And security and simplicity are more important than performance in our environment. For the web server, we originally used Apache and Lebrick and then Lite, HTTPD and SCGI and then Nginx and Mongrel and then finally Nginx and Unicorn, which we've been using successfully for many years. Now on the testing side, we use Minitest, Minitest Hooks, Racktest and KappaBara for testing. Now all of our web applications are explicitly designed around a very simple web 1.0 experience that we do use some HTML5 features such as required inputs, date inputs. We avoid using JavaScript as much as possible, only using it when there is no other way and we absolutely must have dynamic behavior on the web page. Most commonly, this is for input forms that can accept an arbitrary number of line items. We use so little JavaScript that we manually test the JavaScript that we do use whenever we modify the related code. Now all of our applications run in the same stack described here and use the same library versions and that makes it much easier to switch between applications during development. Whenever we wanna upgrade a library, we run one command which runs the tests for all of our applications using the new library version and we can see if anything breaks. This command can also be used to test on multiple versions of Ruby and all of our applications pass their test suites on Ruby 2.3, 2.4 and 2.5. So after upgrading libraries, we may decide that we wanna use the new features that have been added to the libraries and this is generally done starting with our simplest application and then applied to our remaining applications in order of complexity. A couple of examples of this are when we switched to using frozen SQL datasets, databases and models and when we started using Refrigerator to freeze all of the Ruby core classes at runtime. Now when you choose to use Ruby without Rails, you are probably going to limit the number of developers that you can hire that already have experience with your stack. For some companies, it may be easy to find developers that already know Rails. However, we attempted to recruit developers at many different experience levels and specifically highlighted the fact that we use Ruby and we did not even have a single person apply with any Ruby or Rails experience. So Rails' popularity manage really doesn't matter to us. Now the good news is in our limited experience, a Rota and SQL based web stack is easy for new programmers to learn. Now it is unwise to extrapolate from a sample size of one but our current developer had no professional programming experience and had never programmed in Ruby before we hired her. She was able to quickly become productive and implement new features using SQL and Rota and I think if we were using Rails, it probably would have taken her substantially longer in order to become productive. Now I am the department's information security officer so one specific focus area for me is security. As you would expect, we tried to protect against the common vulnerabilities in web applications. We had to mitigate cross-site scripting using Rota and Ruby to automatically escape output in our templates. We protect against cross-site request forgery by using Rota's route CSRF plugin and enforce the use of path and action specific CSRF tokens for all requests that would modify state. Many security vulnerabilities in Ruby web applications stem from the use of unexpected parameter types that are submitted by an attacker. So we try to protect against these unexpected parameter types by using Rota's typecast params plugin and SQL type conversion to ensure that all parameter inputs that we're accessing are of the expected type. We protect against SQL injection by using SQL to construct all of our queries. We do not have any raw SQL usage out on time in any of our web applications. We use a restrictive content security policy in all of our applications to mitigate possible browser issues in case an attacker is able to exploit a cross-site scripting vulnerability. Now we go further than that. We take a defense in-depth approach to our web application security. So a lot of our security planning starts by assuming there is an SQL injection or remote code execution vulnerability in the application and thinking about ways of implementing features that would make that vulnerability more difficult to exploit and to limit the damage it could do. So all of our applications run with separate operating system users with reduced privileges and with separate database users per application. For the applications with public-facing components such as the public parts of our recommendations and recruiting systems, those run as separate operating system users with even fewer privileges and with separate database users that are only granted the minimum access necessary to perform the public-facing functions. So if there was a vulnerability in our public recruiting system, an attacker may be able to exploit it, but they would not be able to use it to change their exam or their writing assessment scores because the database user does not have the necessary privileges to do that. Likewise, a vulnerability in our public recommendation system could only be used to add responses. It could not be used to update or modify existing responses. We use security-definer database functions in our systems to grant specific types of database access. For example, password authentication for users in our recruiting system uses a database function that accepts the password and returns whether the password matches the stored hash. But the database user does not have the ability to access the password hash directly and therefore you cannot export the password hashes to perform an offline attack on them. We also use security-definer database functions in our tests when setting up database state when the database user does not have the necessary access to do so and this allows for fully transactional testing even when the different database users are used. We run all of our applications on our own hardware in isolated subnets with strict ingress, egress and loopback firewall rules that limit the types of connections that can be made to and from the servers. Now we use per-operating system firewall rules. So operating system users for our internal web applications are allowed to connect to our custom authentication proxy but applications for our externally accessible applications are not allowed to do that they're restricted by the firewall. Now to mitigate arbitrary file access and remote code execution vulnerabilities we run all of our applications CH-rooted. So our applications are started as root and after the application is loaded but before it starts accepting connections it drops privileges, well first it CH-roots to the application's directory and then it drops privileges to the application's operating system user. So in addition to limiting access to the application's folder we also use file access permissions to ensure that attackers cannot read any sensitive configuration files that have secrets or other sensitive information. Now CH-root is a very helpful security feature but it has some problems using it with Ruby and in Ruby the main issues with using CH-root are that it does not work well with runtime requires and Ruby's autoload feature specifically is a type of runtime require. Now the use of autoload has been strongly discouraged by mats for many years but popular Ruby libraries such as Rack and Mail still use autoload and that complicates their usage in CH-root environments. Now to make it more difficult to execute blind return-oriented programming attacks that are based on exploiting consistent memory layouts we have each of our unicorn worker processes exec after forking. So all of our unicorn worker processes have unique memory layouts. Now this is a fairly large memory cost but we are writing all of our applications on a leftover server with only 256 gigabytes of RAM and we're currently using about four gigabytes for all of our applications. So for us the additional security is worth the extra memory cost. Finally to make exploitation more difficult in general and to reduce the kernel attack surface for privilege escalation we limit the set of allowed kernel system calls for the applications to the minimum that they need to function. So none of our web application processes are allowed to fork, exec or send signals to any other processes. And if the application process does not need to accept uploaded files it is also restricted from modifying or creating any files at all. Combined these security controls make it more difficult to successfully exploit our applications and make it more difficult to use a successful exploit in order to attack other applications and systems. Now there is definitely a trade-off in some of these restrictions complicate development and cause problems with certain libraries that you need to work around. And this also required modifications to a few Ruby libraries to allow them to work in CH route and fork plus exec environments. So after implementing these defense and depth security features and using them in production for the last 18 months here are my recommendations on whether to consider implementing them. Consider whether you have data that is worth protecting. How sensitive or confidential is the information that you deal with? If you're dealing with anonymized data or public data sets maybe the data does not warrant the use of these security controls. Consider who can access your application? If you're only designing an application for internal organizational use and it will not be accessible from the internet then the risk of attack is lower and maybe in that case your effort is better spent on providing an improved user experience. Consider how much control you have over your application's environment. If you're running on your own hardware or virtual machines you may be able to use most or all of the security features. If you're using a platform as a service provider your options are going to be limited into what the provider supports. All right, if your application is accessible from the internet and contains sensitive or confidential data my recommendation would be to first look at using multiple database users restricted database permissions and security definer database functions to limit the possible risk of exploited SQL injection vulnerabilities. If you have the ability to configure firewall rules I would recommend doing so. The initial implementation is fairly easy and the ongoing maintenance costs are low. If you have memory to spare and the increased memory usage is not a problem then consider implementing 4 plus exec. Now if your application or the server it is running on has any special access to any other applications or systems that are not accessible from the internet or security is a high priority compared to the ease of maintenance and you're running on your own hardware or virtual machines then you can consider a priv drop, CH route and or system call filtering. So I'd like to finish this presentation with a few opinions and first my opinion on what a successful government IT project needs. It needs executive management that is willing to try new approaches. So Grace Hopper one of the creators of COBOL stated that the most dangerous phrase in the language is we've always done it this way. Considering the track record for government IT projects of being delivered late over budget and full of bugs if they work at all I think that new development approach is worth trying. Next the project managers need a deep understanding of the technology used so they'll know what problems the system has and how to fix those problems as soon as possible. The developers building the system should be discussing the requirements directly with the system stakeholders with an eye to reducing the scope of the system to the minimum. Developers building the system should be responsible for maintaining the system when it is in production. This encourages them to make maintenance easy and design the system to be easy to use so they don't have to make any modifications later. And finally all systems should start by developing the simplest possible system that handles the most important need making that system work well and gradually adding features to the system to not run a foul of Gaul's law. My second opinion comes from a lot of personal experience and that is that Ruby is a great fit to build the types of systems the government needs at least for those systems where the runtime performance is not critical. Ruby makes it very fast to develop the systems quickly and get to a working state. See when you have good tests that cover most of the system's functionality it's fairly easy to maintain Ruby web applications modify them as requirements change though a lot of that does depend on your choice of Ruby libraries. My experience is that work external requirement changes cause much more code changes than refactoring changes. So the limited ability to use automated refactoring in Ruby is not a major issue. Ruby is easy to learn and easy to teach to new programmers. And this is especially important for government work since in my experience at least applicants for government programming positions will probably not have much experience with Ruby. Importantly, Ruby keeps programming fun. I think the importance of this is definitely underrated. Government work does not pay very well so having a fun language to program in can help retain talented staff. After being responsible for all programming for government department for the last 15 years versus the developer and now as the manager I can confidently say that Ruby is a great choice for application development and I highly recommend it. And that concludes my presentation. I'd like to thank all of you for listening to me. If you have any questions, I have 45 seconds. Thank you. Four.