 All right, so thank you for coming. This is the Pragmatic Lessons of Rails and Ruby and the Enterprise. I wanted to start off with mentioning, so I work for Cerner and I doubt anybody in the room that at least anybody who doesn't work for Cerner knows what that is. Is there anybody who knows what Cerner is outside of? Oh, so there's a couple. So, Cerner is a healthcare company. We are, kind of our tagline is that we believe healthcare is too important to stay the same. So we are a global healthcare company trying to facilitate and change the way healthcare is delivered, specifically in the U.S. that means quite a few things, but we spend a lot of time working with hospitals, clinical providers, networks, et cetera, providing them various technologies. Most of that you would see with things like electronic medical records and so on, but part of what I work on is kind of a newer frontier for us which is population health and so that is about trying to focus on the care of populations as a whole and trying to adjust the way that we approach healthcare, especially in the U.S. where we're trying to change from a healthcare system that's mostly motivated by pay for service where we wanna move that to pay for care. My name is Nathan Byer. My Twitter handle is N Byer if you're so inclined, it's not all that interesting, but feel free. I've been with Cerner for over 17 years, so that probably makes me old, I guess. I've been working in a variety of technologies over that period of time. I've been doing open source development for most of that period of time. I had the glorious fortune and I guess misfortune of being a member of the Apache Harmony Project, if anybody knows what that is. It was a brief flame to attempt to create a open source JVM and the coolest thing that we got to do was be part of Android and then now we're being thrown out, so that didn't work out so well. But anyway, so what I'm here to talk about today is how we've utilized Rails and Ruby at Cerner and to talk about that I wanted to give a brief history of what web application development at Cerner's looked like and to start that off with, we kinda have to start off with the beginning, which is around 1999, which also happens to be when I started working at Cerner. I started at Cerner straight out of college. I've got my fancy computer science degree. I know absolutely nothing about software engineering or what it really means to build software. And at this point, Cerner really isn't doing anything with web development. We are doing a little bit, so they've been experimenting with this initial kind of webification of PowerChart, which is one of Cerner's solutions. They called it gloriously enough web PowerChart. And because all of PowerChart is most of PowerChart, at least the end user portions of it are C++ on Windows desktops, then natural inclination was what do we do that works with that? So, started building C++ DLLs and then throwing them into ASP. And when I say ASP, I don't mean that cool ASP.NET stuff. I mean the pre-ASP.NET world where you did just this strange syntax. But we didn't know anything about web development at the time. And I don't know that many people did, but essentially most of the code looked like this. It was an enormous amount of code written in C++ DLLs and it was literally writing HTML and C++. The amazing brainstorm that I had, which wasn't I guess so unique, was that we probably shouldn't put our HTML in the compiled code. We should actually put that in the ASP and separate our concerns. It was an amazing feat and I started to feel like I was a key engineer at the corporation. So, it turns out that really wasn't a thing and not too many people really wanted that and we didn't really even know what we were really doing. And so, as things evolved years later, we started utilizing a lot more Java as everyone kind of does at large enterprises. And so, when it comes to Java, at that time Java was still a little, when we started using Java at Cerner, it was still a little bit of are you sure you really wanna do that? It hadn't quite got to the point where Java was the thing that nobody gets fired for using. And so, we'd kind of worked through some of that, but then we'd started doing various amounts of Java development using doing web development. And so, we got pretty good at doing some of that. We were obviously using struts, just like everybody else. So, this was what you would call Java EE, or back in that day it was Java 2 EE. And at that time, I'd been doing an enormous amount of Java development. So, this was fairly familiar. Anyone who's ever seen servlets and struts, you know what this looks like. It was just, it was kind of a mess, right? And this was the beginning of what, I think our keynote speaker referred to as the XML setups. This was even before then, right? We hadn't quite got to, like let's configure everything with XML. As that evolved, there started to be some offshoots of things. So, around 2009, we had some teams, specifically this group called Cerner Health, that was more focused on consumer oriented stuff. And so, they started experimenting with Python and Django. And so, in 2009, this was a completely valid choice, not that it's not valid today, I guess, but it's just a different situation. And so, they started experimenting with Python and Django, and this was a very interesting time that they were kind of a smaller group at that moment, and they were kind of off on their own, doing their own thing. There were still lots of Java development happening, lots of struts, and by this time, we were talking about struts too, and things like that, and all of that pops up. Sort of concurrently, I kind of got this weird assignment to go build a customer facing pseudo e-commerce store. And so, this is what we called the Cerner Store. And at that time, I wasn't completely aware of what was going on with the Python world, and I didn't really quite understand what these guys were doing, so I didn't collaborate too much with them, and this was kind of set up as somewhat of an experiment. So, I just spent a moment and said, well, what's the rest of the world doing? And maybe I should try that. Well, one of the things that I noticed the rest of the world was doing was Rails, and it was like, well, let's just give that a try. This is a very isolated piece of software. I can kind of go do whatever I want to go do. And so, at that moment, we started using Rails, and it was me, and then I got to hire a couple engineers who were straight out of college, and they had no experience either, and they were like, this seems interesting and cool, so let's try doing that. And so, that's kind of where our adventure began, and we started doing a fairly standard Rails application. It was simple Rails, we were using Ruby Enterprise Edition, we were using Passenger, we were using MySQL as a data store, it was a fairly simple monolithic system. And so, that worked fairly well, but over time, we started to see that as our needs, as a company started changing, we actually needed to do much more kind of what I would call cloud-based development, where, previous to this, almost everything that we've been doing was kind of packaged software. So, this is your traditional enterprise delivery of things where you package up all of your software and you give it to your client, and they take it whole, or we can go run it for you, but we run it for you in a kind of isolated way. And so, as that evolution has changed to more of this software as a service, we started needing different technologies. And this is where then we popped up into, we needed to start doing mobile development. So, in about 2012, we started needing to do mobile development. Well, what we found was that we didn't really understand that with a connected mobile application, you really still need an application on the server side that kind of connects to your application, say, on your iPhone. And so, I kind of got wrangled in and I kind of threw out that we should just start using Rails because I've been using this before, and I think this would fit in a really nice niche spot here, which is we're gonna build some web services to facilitate what our iOS applications need. And so, it kind of fit a nice little realm there that we could kind of quickly build these applications and it was very productive and easy to use. And so, we started building lots of basically restful services to fulfill our iOS apps. Behind the scenes though, this is a very non-traditional kind of system in that there's a lot of the Rails applications themselves didn't connect to databases directly, they interacted through services. And I'm partially to blame for some of this, but for some reason we decided to use Thrift, which if any of you are familiar with this this is a binary service protocol and RPC mechanism. I would not suggest it to anyone. But so we built all these internal services using Thrift and so you had your Rails apps doing that. At the same time we also started realizing we need to figure out how to do deployment automation much better and so this is where Chef came in and one of the big I think drivers for Chef was it also used Ruby in terms of the way you defined recipes and so on and that. So it was actually kind of serendipitous for us to be like well we've got all these tools but if we can use some of the same languages it kind of helps us work through things. And then probably six to nine months later I got tacked with working on the Population Health Initiative which is what I'm partially responsible for now and we became responsible for building what we call Healthy Intent and Healthy Intent is our platform for providing population health services. It's a software as a service system. Part of that is delivering web applications and services and so it was fairly natural at that time to say let's just start using Rails again. We'll build our web applications using Rails, we'll build our APIs using Rails. Behind the scenes instead of using Thrift, we dropped that, we still needed to use a lot of Java because we build a lot of internal tools using Java and unfortunately or if this is your kind of thing we have a lot of big data and that really means you've got Hadoop and if you're working with Hadoop you're really working with Java. As much as you want to not work with Java if you work with Hadoop you're working with Java. So to build services on top of HBase and those things we use Jaxrs. On top of this we also defined what we call Blue Steel and some of you will get the joke there but we defined what is essentially a HTML, CSS and JavaScript framework for building applications quickly. So to some extent it's a little bit like Bootstrap. It's essentially lots of components that have these that we can easily pull together and kind of build up an application and the style just works, the HTML just works and the interaction kind of just works. It also provides us with consistency. So that's kind of where we are at today. That's kind of the current state of what we're doing. So we use Rails to do lots of web application development. We use Rails to do lots of public service development but it's not everything that we use. And so I want to kind of point out some of the things that I think are somewhat unique to our environment and as I've kind of spoken with people I say these are seemingly unique because I don't think they are totally unique. I think people just don't talk about them because it's not hip or cool in some circles. And so one of the big things is that we really use Rails as just a single tool in our tool belt. It's really a component in a very distributed system and so this is kind of a very simple generalization of the way that healthy intent would look if you were to think of it from the top down. And so we have lots of Rails instances running that manifest as individual applications and those applications interact with lots of services. And so those services that they interact with will be Java based services. Some of them we've actually built using Rails because they're sort of a simple reference service where it's really just I want to have some kind of a specific domain of modeled and then I need a little database and then it's a nice little service and that builds it very simply there. The Java stuff, we've got a lot of HBase and Solar and we use those in parallel so this diagram doesn't do it justice but we use, if you've ever had to use HBase, it's very interesting for doing key value lookups but it's really not good at anything else and almost everything else you need to do is that anything else. So if you need to search for something, do you find it, you need an index, right? Normally in a SQL kind of world where you use indexes and I do select where X equals this, you do lots of where statements, you don't get that in HBase and so we use Solar to kind of help with that. In the wild you would also see a lot of people use Elasticsearch in this kind of fashion. We also do have cases where we use Solar as kind of both, it is the data store and the querying mechanism to it. It's not super important for this kind of context but obviously underneath that we've got this huge, massive big data processing system and I've simply trivialized it. Also because I really don't like big data, it's just, it's not fun. So if you can avoid it I would suggest it. I unfortunately can't. And so to kind of give a little bit of scale in terms of what this really means, at the top there I've only got little four boxes but within healthy intent that really represents I think something like 15 to 20 actual projects. Below that we're probably talking more like 20 to 50 projects and then it kind of blossoms like that. Across Cerner there's probably 50 to 100 rails projects that are all completely independent doing all kinds of interesting things. And so we've got plenty of usage at scale. So one of the, one kind of unique aspect is how we talk about security or at least some of the requirements of security. I don't think security is anything unique in terms of rails development but we do have some specific requirements that I haven't seen manifest elsewhere. I'm sure they do, which is not talked about but we have this requirement around user event auditing. And so in a clinical setting when you have users working on some software and they're interacting with a system that has healthcare data in it, you have to know exactly what they're doing every step of the way. So every step that they do has to be audited and that audit event has to be made available to the people running the system. So in this case it would be like hospitals and physicians, clinics, et cetera. They have to be able to go back and say, I need to know what this user did, when they did it, what they saw because they could have been doing something wrong. Legally, you can get into situations where for malpractice you have to say, well, can you show me that this person didn't see this chart? I need to go look and do those kind of things and so we have to actually have a complete system where every single action that a user can take gets logged and audited and so we store that auditing and so this is kind of an interesting security feature where we have to have this whole system where we can spawn off these events and say, oh, this thing happened, this thing happened, this thing happened to be able to track that. It's not an especially difficult or onerous thing to do but it does add a lot of complexity to our Rails code and that the application tends to be where you implement the auditing capture because that's where you actually know what the users are doing. At lower levels it's all system oriented so you don't know what the user's doing. So this adds a lot of interesting complexity in that you've got to figure out kind of interesting ways to add DSLs to your controller or model objects such that you could say, well, as a byproduct of this action, submit a view chart audit event for this user and kind of just capture it simply. Another kind of interesting security feature that we've been kind of experimenting with a lot of in the past few years is really about federated authentication. And I think people have seen this before when you've got, I've got my application and I don't have my own identity. I let you log in with Facebook or Google or Twitter, et cetera. I kind of federate my identity. In our world, federated authentication it's the exact same thing but instead of it being Google or Facebook, it is client X has this SAML provider that's built with Active Directory and they've kind of got this weird system for doing login that doesn't really work but we do SAML, so it'll all work together. If you know what SAML is, hopefully you know why I've got this little sarcasm going there, but if you don't know what SAML is, if you ever see it in like an RFI or something or something like, hey, can you do some SAML, like you just leave. Don't get involved with it. And so we've had to build these, this whole kind of federated session system where we completely separate that from our applications and so we kind of have a slick separation of concerns now where we have applications that interact with a authentication session service and all we know about is that you have an authenticated session. I don't really care about how your identity was logged in and what you meant with it. I just know you're a valid user and I can deal with that but then this poor session services, they've got to deal with integrating with all of these variety of identity providers and none of them do it the same. It's all a terrible, terrible experience. So another piece of security that I don't think is completely unique but it is a variation on what you might generally approach. So when it comes to authorization or access control, you'll often hear about the terms maybe like role-based access control where you define this person is a role like they're an owner or a publisher or a creator and you have permissions that you do based on that. Well, within the healthcare realm, what we've got is we have a variety of things where it's sort of role but roles are much more arbitrary than that where and so we kind of use what we call group-based. You can define groups to be roles but you can also kind of just define groups to be like I'm in this really cool group so that means I get to do these really cool things and so that happens quite a bit but what makes that much more complicated is that authorization also has to be based on data in the system. It's not and this is where the more complicated access control is that it's a user's relationship to data not a user's role that tends to be the more complicated and more important security. So a user's relationship to a patient, so say your doctor's relationship to you, they get to see your chart because they have a relationship with you. Another doctor doesn't get to just see your chart because they don't have a relationship with you and so there are hundreds of these kind of database access controls that you have to try to model and you have to try to work through where it creates this fairly complicated situation where you have to have layers and layers of complicated code where it's like, okay, are they in this group? Can I let them do this and then do they have this relationship with the data? Should I allow them to do that? And it's especially complicated because oftentimes that relationship isn't direct, it's indirect and so it'll be a user's relationship to something like an organization and then that organization's relationship to a patient. So I get to see that patient's data because that patient's related to the org and I'm related to the org and so then we both get to do. And this is an extremely kind of painful and complicated thing that, again, this manifests in our Rails applications because that's where we do a lot of our security because that's where the user meets the data. The other, I think, kind of major aspect of things is really what I call development ecosystem and this is really oftentimes in a smaller environment it's kind of a byproduct of just doing development but it's something I worry about a lot. It is really how engineers are productive and how they go do their job. What are the tools you have for doing builds? What are the tools you have for accessing libraries? What are the tools for documentation for all of the libraries that you're building? How do we make all of that consistent? How do I make sure you're using the right libraries? How do I make sure you're using the right code style? Things like that. One of the things that I think that pulls in for us is that so we have an enormous amount of, because we have so many projects, Rails projects, that naturally is gonna lead to lots of gems and so we've got hundreds of internal libraries that we have to manage and maintain and so obviously we have to have our own gem server. And I don't know if you've ever had to deal with trying to run your own local or internal gem server but this is not something that the Ruby community does well. And this is honestly something that I've kind of bristled against for a very long time. So I mean, for a long time I was very much a Java developer and as much as it's kind of gross and things like Maven are kind of gross, Maven nailed it in terms of dependency management. They really work through all of these concepts of I've gotta have a repository and I've gotta have namespaces and names for artifacts and versions and then I'm gonna define dependencies and define all this tree. Ruby gems did a little bit of that. They've got an inkling of that. Bundler comes in and tries to do some more of that and Bundler is evolving. But it's still kind of a difficult patchwork of things and so this is still a very painful piece of like, I can set up my own gem server but Bundler with multiple gem servers is kind of weird and it'll try to warn you about oh, there's this name that's the same in these gem servers but I can't help you fix that and so this is kind of a very painful part that I would like to see help in the community in terms of being able to have more of a distributed mechanism for dealing with gem management. It's very, very painful. One part of our development ecosystem that's kind of important is what I would call builds and so this kind of encompasses a lot of things but builds are kind of important for us in that. So as a, I don't know, I hate, again I use the word enterprise but as kind of an enterprise we get tacked on we've got lots of these regulations and processes so Cernar is an ISO 9001 certified, we have to deal with the FDA, we have to deal with HIPAA, we've got ethical obligations as well as our legal obligations, we've got all these things and so we have lots of processes that are very general and generic that talk about, okay, if you're gonna do development you have to have all of these things that you do in development to prove that you're doing it right, that you're auditing it, that you can recreate it. We have frequent visits from not so friendly people that come in and say like oh, you're not doing this step in your process right and it's extremely painful and that those people come in and you pay them to come in and tell you you're terrible. If you've never dealt with kind of regulation and process it can be a difficult thing and so some of the topics that come up is like so when we talk about like when you just work with the code, when you're finished like you can't just say like I'm finished and then it just works right, we actually have to go through and say like we have to make sure it's actually tagged correctly, packaged correctly, we have to make sure it's tested correctly and you have to prove all of those things and so it isn't so much that you can just test it and that your unit tests run, you actually have to have the results of those unit tests and you have to be able to prove that you could recreate them and they have to be done and all that things and so kind of as a side effect of that we've created this kind of tool we call rollout and it's a little bit like a really poor man's maven but essentially what we've built is each project gets a description of the project we call it the project yaml file, tells you what the project is, its names, tells you where its source code is, tells you where its JIRA is, tells you all kinds of information and then we've got a little tool that kind of wraps around things like bundler and RubyGems as well as our doc generation as well as running our spec and test unit and then all kinds of other things that we wanna pull in so pulling in we wanna run breakman audits, we wanna run a dependency report so we kinda have to just say here are the real dependencies and do all of these things so that then we can run our builds off of our tags and then archive it. It seems like it's kind of a trivial thing but it can be kind of painful and expensive if you don't just have it and it doesn't just work and so speaking of dependencies one of the things that we've found is working with Rails and I think a lot of people already know this is that you've got to stay up to date and in our world that can be interesting in that not everyone is moving very quickly and releasing very quickly so if we talk about Cernar's kind of older software so software that's been around for 30 plus years they've got development and release cycles that measure in months and almost a year and so when Rails updates come out every few weeks whether there's patches or fixes or stuff like that it's kind of a whole new world and then if you add in all of the dependencies that you start using all of your dependencies are releasing new code all the time and then they're also releasing fixes all the time and if you don't stay up on that new code it'll break and so you do not wanna wait till it's like oh they released 2.0 I was like I just don't have time for that like it could be three months later and they released 3.0 and you're like oh I've got some time I'm gonna go do that upgrade it's almost impossible it was like the lightning quick changes that can happen in these environments are kind of almost difficult so you've got to stay up to date so fortunately we've gotten pretty good at this almost all of our Rails applications in the healthy intent environment I was talking about all running four to five or six I think it's four to five but we've kept them up to date very quickly we move as fast as we can and also at the same time I would say we like to trim them as fast as we can you've got to keep a handle on the dependencies you cannot just let everyone decide to be like oh I think this is a cool one I'm gonna go do this and pick up that both from a practical and pragmatic standpoint as well as a development control standpoint you've got to vet your dependencies so we don't just go pick whatever we want you can't just go out and say like oh I'm gonna start pulling this down because it's the latest coolest thing right yeah like in JavaScript land like that's even worse I don't know I don't get involved too much in that world fortunately but I can't imagine dealing with that but you've got to vet your dependencies so we spend a lot of time actually trying to understand most of it is open source dependencies so you have to go look at that dependency as like if it's a community is the community actively maintaining it are they still developing it right you can't just if it's one dude that's a risk I can't take a risk on a dependency just because it's super cool right it actually has to be valid if it's one guy there's a risk that guy's gonna fall off the earth right or he's just gonna get bored right and the mitigation for that for us is if that guy falls off the earth we own his code right so that's a challenge that you think it was like because you won't necessarily be able to remove it instantly so the moment that becomes lost you own it and it's yours to deal with so this is something that we spend a lot of time with kind of talking about so try to do this quickly some of the lessons you have to be aware of culture shock so as cool as Rails and Ruby are people will hate it just because it's different and so if you got Java developers you'll see things like this a lot where like I didn't even know you could do this where like you could do private and then public again this was like this was really cool I thought that was awesome but then just things like why do you need a factory and the best part was like this was a service factory it wasn't in a library it was in a Rails app so it was like creating an abstraction that just you didn't need the abstraction like everywhere you needed to do you could have just done new for the service and it would have been fine and then hidden in here too is this interesting you see this dot absent like they actually built an extension to like string and nil and like all objects to add the absent which is really just the nil question mark feature you know function that's already built into Ruby I was like no I just don't trust it I'm gonna build it all myself like this happens I don't know how many times we've seen this the other one is if you run into we run into this world where like if it's all static types right people just are afraid of dynamic typing I don't know I don't know why you know I've never kind of understood this it's just like if I'm in Ruby it's just where it is if I'm in JavaScript it is its own thing right these are all their own thing but people will spend a lot of time and energy writing hundreds of lines of code on like is this type string is this type hash is this type like to go through all of this stuff is like so you have to kind of be prepared to teach them that duck typing is okay and that loose typing is okay and that like it might just be nil it's okay all right those there is serious fear about this and as an aside I've tried to there also seems to be a weird fear of this in the big data world too is that there's this fear of like text where it's like CSVs and JSON or just plain text versus I've got to have proto buffs or Avro and blah blah blah and I'm not I haven't completely decided this but I think that it's pretty much the same thing as static typing and dynamic typing right everyone who's like I've got to have these proto buffs and Avros is like these are all static typing people right it was like if you can just deal with JSON and CSVs you know you're probably just a dynamic type and I think you'll see that too if you go dig into these any of these big data conferences of technologies people are using Python right all they use is CSVs and JSON right people that are using Scala right it's Avros and proto buffs it's a very interesting dynamic and it's like you've got to have your types and I've got to have my data typed and it's one of the other pieces that and this isn't just a Rails thing but I've seen it pop up more in Rails because of the way that when you work on a Rails project it kind of encapsulates things is that there's a lot of misconceptions around reuse and so I see three things that I often bristle against and so one of them is what I call micro frameworks and this is like where it's related to everyone wants to write infrastructure I don't know and maybe this might be just me talking about a Cernar culture thing but like you are a cool guy if you're writing the architecture and the infrastructure right like you're not so cool if you're writing features functionality so you tend to see this like I'm going to create some frameworks within my Rails app that are going to make developing this Rails app be super awesome and sometimes this is creating abstractions over the existing abstractions and kind of doing things the other piece is like it's lots of do not repeat yourself when it's really not repeating yourself all that much I think reuse can be tortured and overused and it really doesn't work at scale and what I mean by scale is when I talk about healthy intent which is the organization I work in so and I'm responsible as the principal architect for the direction of about 300 or so engineers and architects when you're dealing with 300 engineers and architects reuse is a little overrated because each individual person thinks they are dealing with reuse and that they are creating the best thing but what happens is you get 300 snowflakes and they're all special and they all want their own frameworks and they all want to like no I'm going to do it this way because it's super cool to do it this way and I'm going to make sure this Rails project does it this way and then what I end up seeing and dealing with is that I've got this Rails project that's a snowflake and this Rails project that's a snowflake and the only difference between them is that one of them has domain model X and one of them is domain model Y but all of their code is completely different so I can't move people around and be like hey can you help me build this feature over here they're like no that thing's crazy looking I can't believe they went and did that but if they were to just been doing it my way it'd be totally perfect I was like well like that person over there is saying the same thing about you and I was like so this is something that happens and I don't know that it's necessarily enterprise problem but at scale it happens a lot all right so I don't know if I've got any time but if there's questions I will come answer your questions you can come up we can chit chat I have three little reservations for the Cernar party at that you can get into if anyone really wants to get in I can give you one if you ask me an interesting question tomorrow as when the booths open up in the exhibit hall tomorrow they're giving away a raffle for a Jackstack gift card which if you're in Kansas city you can go to Jackstack and get your barbecue fixed I believe it's like a hundred dollars so it's like more ribs than you could deal with and then I believe it's Friday we're giving away a Phantom drone which is like a thousand dollar drone and it's super awesome so I would suggest you go to the booth and get that there's also people walking around that have the Cernar engineering hoodies because no one else is wearing hoodies you can go chat with those people about Cernar stuff again you can come up and talk to me thank you everyone I appreciate your time