 Everyone, welcome back to theCUBE's coverage of Ansible Fest 2021. I'm John Furrier, your host of theCUBE. We're here at Eric Pettington, Director of Solutions Engineering and Mike Tadaro, Senior Epic Cache Consultant at Sapphire Health. Gentlemen, thank you for coming on theCUBE and chatting about the wave of cloud, cloud native and Sapphire Health and Ansible. Thanks for coming on. Thanks for having us. Thank you. So let's get started. Can you guys just briefly describe Sapphire Health and what you guys are doing there, the consulting services, the trends that you're seeing. Just take a step, a minute to describe the environment Sapphire Health and what you guys are doing. For sure, yeah. So Sapphire Health was a consultancy that was founded by the CEO back in 2016, Austin Park, who also serves as a CTO for some healthcare organizations because he was having difficulty finding an organization that really specialized in Epic infrastructure. So you might be familiar with some of the large players in Epic Consultancies, but they are typically focused more on the application side. So configuring like the ambulatory clinical system or something like that. And there really wasn't a solution that he could find in the market for an organization that was focused on Epic infrastructure and some of the more technical components of managing an Epic technical ecosystem. So Austin founded a team. Mike was one of the early folks to join. I joined a little bit later, but he put a team together to again, really focus on the technical components of an Epic implementation. And since then we've been providing managed services for Epic infrastructure for a number of organizations. We've been focusing on platform migrations from, for example, AIX to REL for Epic organizations. And we've been focusing on some growth areas as well in the cloud. Epic Systems is now able to be hosted on the public cloud. That's a relatively recent occurrence. So we're working with some organizations in that space as well. Mike, anything you'd add there? No, I think that pretty much covers it. We've spent a large fraction of our effort making sure that we're engineering solutions for these clients that move them in the directions towards cloud readiness, towards containerization, automation and those sorts of things. So that I think Eric's description spot on. So you guys must be busy. I mean, I can only imagine the action happening right now as people realize that the pandemic specifically, two areas that we've reported aggressive growth on was public sector and healthcare. Both were under massive strains of pressure to get faster. Can you guys just weigh in real quickly on what you guys are seeing? Because, and how's that impacted your consulting services but also the customer? What's going on in their minds? Absolutely. We had some customers very early on in the beginning of the pandemic where we were given the cadence of updates coming from Epic, the needs for growth for those customers where both in ICU surge capability as well as just general admittance, there was a flurry of hardware purchasing, provisioning set up and increased cadence around patching for various pieces of the Epic environment including Epic code directly, all of those things, all those, the tempo of all of that increased once the pandemic began. And we spent a significant fraction of time trying to find better ways, faster ways to engineer what we were already doing for clients simply so that we could continue to keep up with the surge in demand without requiring an additional surge in investment in people where it wasn't necessary. Obviously some growth was necessary but we wanted to help our clients get the most out of what they already had so that they could spend that money where it was needed to help patients. Yeah, awesome, great stuff. So we're here at Ansible Fest getting into the action, it's all about automation. So I got to ask you guys, what led you to start exploring automation solutions at Sapphire Health? Yeah, so there's quite a few reasons. I would say the most critical is that we've been providing managed services to organizations around infrastructure management for some time. And as you can imagine, infrastructure management has some repetitive tasks. And I'm quoting my colleague Mike here, but a good administrator is a lazy administrator. And what we mean when we say that is, if there's a repetitive task that's being performed over and over again, if there's an opportunity to automate it, that's going to save us time. But more importantly, that's going to, apologies, lights here. Let me move around a little bit, should come back, there we go. But it's going to provide an opportunity for us to focus on more value add services for the client. It's going to reduce cost for the client in terms of the services that we're providing. And I think most importantly, it's removing the possibility for human error or the possibility for error overall. So it's a natural evolution of us observing the time that we're spending with our client partners. And again, it really provides a lot of value to Sapphire as an organization and our customer partners as well. Mike, you want to weigh in on this automation trend? Well, how do you see it evolving? I mean, obviously it sounds good when you want to automate things you do repetitive tasks, but is there more going on that you see in automation that goes beyond just, okay, if you do it three times, automate it kind of vibe. Sure. Automating repetitive tasks is the kitty into the pool. That's how we get, that's how we sell the idea to people who just don't get the concept yet. But there are workflows that really aren't feasible without outside of automation. We tend to think of automation in some cases in this sort of limited way, but automation is really, what we really are targeting with automation is more about workflow. It's less about individual tasks. It's more about an idea, a workflow or a business requirement from its origin all the way through its implementation. So I've got it just the simplest case that jumps immediately to mine. Because I have a new hire, I've got to provision them an account. I need to provision it across multiple systems. I've got to do it in a single sign-on. They need home directories, they might need access, they need building accesses. We need to generate, you got to generate badges for these people. And these are all workflows that are normally disparate. You have to take your sheet to this guy, take your sheet to this guy. Here's my new hire form. Really, what you really want is we got a new hire, everything's checked out, put it in this basket here and let the automation move it through all of these systems all the way across. And that's the sort of thing, like I said, that's a very limited, very simple idea, but that's the kind of thing we really want. We want to get in the door with automation, with simple things, and then we want to teach, we want clients and ourselves to be challenged, to be creative, to find new ways to apply it that aren't immediately obvious. I was smiling because I love the example of the kitty end of the pool because automation is going mainstream. And it used to be kind of, for the geeks who were doing the hardcore stuff who got the whole big picture. Now you're seeing with AI automation moving in and with cloud and a lot more automation happening. So, you know, I can almost see in my mind, mental image of people wearing bubbles in the pool, kind of like going in the deep end, get back over here, stay in your lane. But this is the trend. And I want to get into this because you guys are involved in this epic migration has been talked about. So for the folks that aren't in, say the healthcare space, put a little context around epic and then I want to get into this whole migration discussion. I think that kind of points to some real value proposition. So take us a minute. What is epic for the folks outside healthcare? Sure. So epic is one of the leading EHRs or electronic health records software in the world. It is by far the most deployed in the United States. What's involved in building an epic or performing an epic migration, epic is hundreds of systems. When you think about epic as an umbrella concept, it is servers and end user workstations and all of these things. When we talk about a platform migration, what we're usually talking about is the transactional database. They call it the ODB or, you know, whichever term I think you feel applies best. When we perform on those migrations, we're usually talking about, when we perform on those migrations, we're usually talking about an AIX to Red Hat migration, although you can just do hardware to hardware. Involved in that is a number of things. You're building new VMs, talking about setting up patch cycles, setting up the patching server, installing the various administration scripts that epic provides, installing the software that runs the DB, which at the moment is either intersystems, cache or iris. There's the provisioning of the local security users. There's the configuration of the OS. There's, if you're moving from AIX to Red Hat, you're talking generally about a bit Indian conversion. So, you know, big Indian to little Indian. There's a tool for that. There's a lot of these little stats. And the thing is, is that they're all very, very well-defined and very similar. And so they look identical in many of these cases from one implementation of Epic to the next. And that's not true for the entire Epic stack. Not necessarily, but at the ODB level, this stuff is all very similar. And this is a very right place to automate. This screams automate. And we do this because, I mean, who wants to make mistakes? You know, if you write and build your script and debug it, the script runs, it doesn't make mistakes. I make mistakes, the script doesn't. So we do that and we end up spending less time on these repetitive, unnecessary tasks. We guarantee the correctness of them or we do a better job of guaranteeing the correctness of them. And all of that ends up saving money in the long run. That's awesome. And thanks for the context. I was going to get there on the automation piece. It really sets the table for the automation. Real quick clarification. How much or what kind of software work is involved in a migration? How much? So there's the installation of, you know, you have the, from the installation of the OS and the configuration of the OS, the building and the patch server, the implementation and testing and patch cycling. There's those data conversions I talked about. There's environment refreshes where we copy an existing environment on a regular basis to another environment for things like testing, for troubleshooting purposes or for other reasons. There's more than one database for Epic. There's one big production database. And then there's, you know, you have training databases and you have playground databases for people to work in so they can learn to use the system better. And then there are, I mean, there's a galaxy. Oh man, it's a huge system. Okay, so I got to ask the security question. Sure. The security element is important when selecting automation or how's that factored in? I mean, right now that's super important. Obviously, records are key, but like obviously, where does that fit into the automation piece? How's security? Yeah, I think that's a very important question. And as you alluded to, security is incredibly important. It's very important in healthcare in particular. And in fact, with healthcare, there's a lot of regulatory requirements. There's a lot of requirements that individual healthcare institutions have that we as a partner to that institution need to follow. So as we were evaluating automation vendors and automation solutions, a highly secure system was not a nice to have or like a value add. It was something that was absolutely critical and paramount to being able to successfully automate any of the things that we're doing. So I'll turn it over to Mike to talk about some of the specifics, but as we evaluated Ansible, we saw that it really supported robust security. So Mike, can you comment a little bit more on that? Sure. So there's a number of ways that we use Ansible to help improve the security posture for clients. One of the ways is Ansible playbooks are written to be runnable against a server and then nothing will change unless something is set incorrectly. And this lets us assure that the configuration is where we expect it to be. So we don't get drift, we don't get drift on these servers. Now, remember I said an Epic environment is a lot of servers. If one or two of these servers- Mike, you don't mind, what is, when you say drift, what are you referring to? So when I say drift, what I mean is if there's a bunch of different servers and I as an administrator have to work on one or two of these servers just for little things during the day, I might make a change on one of these servers inadvertently or inadvertently. And then that server's configuration is now slightly out of phase with the other servers, which could be benign, but it could also be a security hole. Having Ansible able to run nightly and continue to adjust these servers back to the expected baseline and in the case of things like Tower, be able to report that these things were out of position. Let us know, hey, it lets us reduce the attack service. First of all, it lets us multiply, like force multiply our attention across this farm of servers. And it gives us that sort of clarity that we know we're doing what we have to do to make sure these servers continue to be safe. That's an awesome service. And that right there is, I mean, just going in manually trying to figure all this stuff out is just a nightmare. I mean, what a great relief that is. I mean, just the alternative is what? More pain and suffering human-wise as the labor and then risk on attack. Because people go to bed, you know? I'm a patient. The thing is on a personal note, I'm a patient too. All of us are. We all have doctors. We all have, we have to go to the hospital for things occasionally. And if we fail when we perform these security audits, if we fail when we perform these security checks, patient data can get lost. It can get sent to people who shouldn't have it. And I'm a patient. I have no desire for my medical information to be available anywhere, but in the hands of my doctor or myself. And that's the thought I try to stay with when I'm working on these systems. I'm a patient. It's not that I'm doing this because I mean, the knock on effects of reducing liability for the customers cannot be ignored or overstated. And they're critical, but ultimately my eyesight is on the patient. Yeah, and having that stability is huge. Okay, this brings up the whole automation thing as it becomes more mainstream. For you guys, specifically it's critical. The systems there, you have to watch farms, you know, all that, all the action happening is a huge system, complex, automations key. How are you guys continuing to push the automation envelope into the Sapphire Health's consulting practice? Yeah, well, as you mentioned, John, yeah, we're really taking a look at the entire technical infrastructure when we're working with our clients. And we are offering fully outsourced managed services for organizations, not just around the epic infrastructure, but things like networking devices, security, and other third party systems. So with that, we're seeing a lot of these things that are going on and we're always evaluating opportunities for automation. There's actually two areas in particular that we're seeing gain a lot of momentum with our customers. And we're seeing a lot of opportunity for automation. The first is business continuity and disaster recovery specifically within Epic. So Epic has very stringent requirements for resiliency as you can imagine. When the system goes down, a hospital can't really do what it needs to do from a billing standpoint, a clinical standpoint. So very robust disaster recovery resiliency standards and solutions are very important. However, there's not a lot of automation that's available either from Epic as far as I know other consultancies. So what we did is we built a script that provides failover automation. So some of the tasks that would be very manual in terms of failing over to your DR solution, we've automated that. And that, again, removes a lot of the opportunity for human error really speeds up the failover process. And so with the customers that we work with, that's something that we provide. Another big area that we're seeing is environment refreshes. So within Epic, there's different environments that are basically all their data is copied over on a recurring basis from the production environment. And the refreshes can have a lot of manual steps involved. So we've found an opportunity and have implemented some automation around environment refreshes for some of our managed services clients. And as we continue to go throughout, building our cloud practice in some other areas, I'm very confident that we're going to see, infrastructure is code more opportunities for automation around areas like that. And you guys got to love the DevOps vibe going on now. Mike, I mean, you guys have seen the movie before in the old legacy going back to the mainframe. So you probably still run into a lot of older systems that still do a purpose. I mean, I have a lot of friends and clients that are working in the big banks and they still have all the old school that does their job well, but containerization and cloud kind of give life to these systems. Because now we're living in this system architecture called distributed computing again with the cloud. It's the same thing. Different, different, different, different stuff though. Absolutely, years ago, almost every Epic client was running on AIX and maybe not mainframe, but more mini computer. And the migration path for almost all of the clients has been to move from those AIX mini computers down to VMs running Red Hat. Or running Linux and the natural evolution of that path is to move out of, is to move at least disaster recovery data centers into the cloud. And then for some clients, the economics say the whole data center to the cloud. So absolutely that path is well forged. It's there. I suspect that we'll see a lot more of clients, even larger hospitals beginning to move down that road in the near future. And for the folks watching who may not have the scar tissue that we have AIX was IBM's old Unix kind of mid range, mini computer, it was kind of client server, was client server, going now, being being modernized. So obviously Red Hat's not part of IBM, but it speaks not just to IBM. This is about Ansible, right? So this is like this action happening here. So this is a case study of pretty much all migrations. It's not just the fact that it's AIX to Red Hat. It's system to the new thing that has benefit. What's your take, Mike, on that kind of paradigm? It's a lot of people going through similar situations, just change AIX to something else. You have a lot of this migration replatforming going on with the opportunity to kind of tweak it and add stuff to it. What's your advice and what's your reaction to this big trend? My advice for this, honestly, my advice is when you're planning these migrations, you know they're coming. Even if you're not in the cycle yet, you know it's coming. My advice is start brainstorming your implementation of automation now. Get your automation into the system as you platform into your new platform because it is far easier to build that entire platform with automation as a critical component than it is to bolt it on later. And you will get much more out of your investment and time and effort if you've integrated it from the very beginning. I would say anyone that was looking to perform a platform migration now and hadn't already begun serious consideration of running automation or had had no plans for automation was setting themselves up for a very long and very difficult road to hoe. And I would advise against it at this point. Great, great insight, Mike. And thank Eric, thanks for coming on. I appreciate your insight here. You guys want to give a quick plug for the company, what you guys are looking to do, hiring, any update you want to share? Because great content you guys just shared here. Thanks for doing that. Take a minute to put a plug for the company. Yeah, I think a quick plug here. Yeah, if you're a talented cache admin, there's not too many mics out there. So we're definitely looking for more mics. But more broadly, we're really looking to expand into the cloud space. We're rapidly expanding our managed services opportunities. And what we're seeing is a lot of organizations have like one ODB admin or one client systems ECSA admin. And what they run into is that person will leave, that person will retire, that person needs to get married and go on their honeymoon. It's kind of a problem. So we're working with a lot of organizations to not just fully outsource their environment, but to provide a hybrid managed service to provide overflow, to provide capabilities, to scale up with upgrades and projects like that. So talk to us. We're pretty darn good at it. As you heard from Mike, we've got a couple of mics. Again, we could use more. So if you are a Mike, please reach out. He could virtualize them. He would just virtualize Mike with, you know, that's virtualization is a huge trend. Data rights, Mike. We need to do that. Are you a body? Are you the real Mike? As far as I know, my wife would appreciate it if you guys would clone me a few times. You know, I've heard horror stories, Eric, around root passwords. Like, who has the root password? Oh, she left two years ago. Kind of situations. This happens. I mean, this is not, like it sounds like crazy, but people leave. I mean. Yeah. I mean, nobody works anywhere forever, right? I mean. Don't be that company where you lose the root password and never mind the ransomware action. I mean, oh my God, must be brutal. Anyway, we could go another thing and on that. Eric, thank you for coming on. Mike, thank you for your insight. Really appreciate it. Thanks for coming on. Appreciate it. Absolutely. Absolutely. It's our pleasure. Continue coverage of AnsibleFest 2021's The Cube. I'm John Furrier. Thanks for watching.