 Live from Orlando, Florida, it's theCUBE, covering .conf18, brought to you by Splunk. Hey, welcome back to splunk.conf18, conf eight, hashtag splunkconf18. My name is Dave Vellante. I'm here with my co-host Stu Miniman. You're watching theCUBE, the leader in live tech coverage. It's two days of wall-to-wall coverage. This is our seventh year, Stu, at the conf. We're seeing the evolution of Splunk from kind of analyzing log files to having deep business impact across the organization, doing more with data. Jim Nichols is here, he's the DevOps manager at Improvata, healthcare company. Good to see you, thanks for coming to theCUBE again. Thank you for having me, thank you. So tell us about Improvata and then love the title, DevOps and the title, we'll get into that. Sure. First, the company. Yep, so Improvata is the healthcare IT security company and we provide healthcare organizations around the world with secure identity management, multi-factor authentication and enable just ubiquitous access to whatever sort of medical systems that they need to get into and we really try to enable healthcare by establishing trust between the medical providers, the patients, the data and do that all securely and seamlessly so that we're not, security's not a part of their workflow, it's just in there and they don't have to think about it and they just get access to what they need when they need it. So I hear, yeah, on your website, trust between people, technology and information. Reminds me a little bit of a certain software company that's branding's all around us today. Seems like there's a lineup between what Splunk does in your company's mission. Oh, there absolutely is and like Splunk, Improvata has a very strong on-premises in the data center footprint and we're expanding that into the cloud and that's where most of my work is is kind of managing those cloud systems that kind of compliment the on-premises appliance and we're looking at how that's going to move into the cloud and what that means and it's very similar to like what Splunk has done with Splunk Enterprise and now moving into Splunk Cloud and we're actually a customer of Splunk Cloud. Everything that we do, that we could possibly do is out in the cloud and not in the data center. And you've got DevOps in your title so maybe bring us inside, you know, what that means at Improvata. You know, usually think about, you know, moving fast, things are changing all the time. It's themes that we heard in the keynote this morning so explain that a little bit. Yeah, so the way, the DevOps model that we follow at Improvata is really like kind of a consultant model where we've got a small team of very senior, very expert DevOps folks and they kind of get assigned out to the agile teams and they're a team member that gets planned into the sprints, planned in what we're going to be doing and really kind of make sure that those deployment events or the DevOps work that we need to do is planned in as part of the normal development work and that consultancy model is really good in regards to Splunk because we run the Splunk infrastructure, we do all the training, we do some of the basic dashboard work and make sure that no matter what the team, product, onshore, offshore, wherever they are, we're all looking at data exactly the same way, exact same dashboards and it really kind of forces the knowledge to get shared throughout the organization across products and how we think about things and so Splunk, you know, DevOps isn't like a tool or a thing or whatever but Splunk is definitely a great like enabling, forcing function to make sure that we are sharing metrics, how the system works, what we're alerting on and all that stuff in a really consistent way. So you know the T-shirt, meh, tricks, have you seen that? I have. What do you think that means? So it's like the same old, same old meh, metrics. So what does that mean to you guys? Do you have new metrics? Do you have a sort of new set of KPIs that you're using to measure yourselves? So I think the meh tricks part is that it's, I don't know, maybe 10 years ago, the IT industry figured out how to get every single metric about CPU, memory, disk, RAM and all the tool, there are a lot of different tools for doing it, you know, Splunk, Xabix, Datadog, others, I don't know if it's okay to talk about other products or whatever but you know, when you get like a CPU alert that goes off, all right, the CPU usage is 92%, is that good or is that bad? It sounds kind of high and you get that alert, you look at that CPU chart and it's like, there's no context, there's no information and you know, you might be designing your system to run it 90% if it's doing some batch processing or something so it's like metrics, it's like eh, you need to get the alert, you need to know what's going on but you really need to like get the insight into what it is and it's why a lot of the stuff that they showed this morning at the Kino was really exciting where you've got the metrics in one place, the logs in one place, it's all in one place so you get that alert and you can look at it and then see what else is going on without having to like jump into a bunch of different systems. How about DevOps? You had DevOps in the title, what is, how do you guys look at DevOps? What is DevOps to you? And where did it come from and where's it going? I think that I've been doing DevOps my entire career since I got out of college and I came out of WPI and was setting like performance evaluation and it's like how do you measure systems, get the insight, how do you make sure they're running efficiently and I think that what I was kind of doing on the performance engineering side kind of intersected with like the agile movement and folks getting into the agile development teams and trying to integrate that knowledge in the metrics and how you're going to run it in production into that sort of product building process so I feel like I've been doing DevOps for a long time and called it different things over the years. You know, for us at Improvata, it's really about enabling our developers to deliver functionality to our customers as fast and as safely as possible. So, you know, we're in the healthcare industry and you know, the systems that we build and integrate and support life, right? Like these are doctors that are using these systems, they have to work 100% all the time. And that adds some interesting wrinkles where you wouldn't really think about doing continuous deployment for the system that somebody's going to get logged into to get into their medical records. You might want to be able to move that quickly if you need to if there's an emergency bug fix but the level of safety and testing that we need to put in before it actually gets into production, that's really where we spend a lot of our time in DevOps is making sure that that's efficient, that that's fast and then when it goes from going from like a test environment into production, if it takes an hour, for us it's not that big of a deal, we're doing like multi, two week release cycles or even longer and so as far as like DevOps, a lot of the movement has been around like continuous delivery and deployment and we kind of use that to optimize like the test build and debug cycle and that way when we know when we get to production that's going to go smoothly and that there aren't going to be any unanticipated problems. How does security fit into this conversation? Sometimes, you know, the buzzword term, DevSecOps, how does Splunk in your practice look at security? Well, so we're a security company and we wouldn't really ever call anything DevSecOps because security is ingrained in a part of every single thing that we do. Walking into the building every day when we badge in, I think about it, our security people like is the building secure all the way into like what we're ending up doing in the system so obviously Splunk is a huge supporter of that so we've got audit trail information on all the systems and we can know not only what you, our system administrators and DevOps users are doing but like what Docker is doing, what commands it runs and really get at a very, very low level of detail and we literally have everything that ever happens on those systems is audited and we've built a whole set of alerts around things that we know about, things that we think might be a problem and we use kind of our expertise in the healthcare security space to then apply that to all our cloud systems so it's like we never have a team called DevSecOps, it's like it's just what we do. It's the first, most important thing that we think about every day is security so that's why it's a little bit different for us but we like some of the ideas and we've started doing some work around automated security testing on the application code, running like static analysis, dynamic analysis, integrating web scanning tools into our CICD pipeline so that it just makes it that much easier and not wait till the end before you ship it or whatever, we have it right in the development process. What's the regime for your organization? The classic development and operations, throw it over the fence and okay, DevOps brings those together but you still got a spectrum of skills and presumably you've got people on some kind of maturity model where you've got sort of newer folks, maybe guys coming out of college like you were several years ago and you're training them and sort of your one unified team at the same time you might have some degrees of specialization so what have you found is the right regime for the DevOps team? Well I think the consultant model that we've established works really well and we've got a very senior DevOps person that's on the Agile team and they may do some of the really tricky bits but once we get out of the part that only us as DevOps can do, we really try to get the developers to do it so a lot of that's like Splunk training, how do you build a dashboard? Here's maybe a simple example dashboard, now you do the next panel, that sort of thing to try to level everybody up and get everybody on the same page. In terms of this divide between Dev and Ops when I actually joined and provided DevOps was in IT, it was managed as part of our SaaS management offering along with a lot of the other applications that IT managed and one of the very first things that our senior vice president did was like, they got to be in development, they can't organize, we're working together, we're all on the same team, we're all doing all that stuff but just mentally, organizationally, get rid of the divide, put them in engineering, I report to the VP of engineering just like the developers, development managers and architects and that's a way we've just get rid of any organizational or thought divide between the groups. Jim you mentioned alerts just now and we've heard it a few times, you get alerts and I imagine the beeper in the old days, now you get an alert on your mobile phone. Where are we in terms of being able to take action on those alerts, have the machines take action for us? Is that an objective that you have? Is that just too damn scary? Your thoughts. Yeah, so my first impression is that it's a little scary. We do have some problems that occur with some frequency, so losing an Amazon EC2 instance happens 10 times out of 100 instances in the cloud on a given month so there's certain types of those failures that we've automated around just because you have to as a part of just doing business in the cloud so a lot of the Amazon like auto scaling groups, all that stuff, we've got a couple of issues that happen that we want to just resolve faster and repair faster. They don't impact customer experience or user experience but we just want to get on top of those sooner. So we've started to automate some of the very thin, small, carefully controlled use cases so that if the alert were to go off spuriously, I know it's not going to then take down a system that was running and finding good. A false positive is something you get. Exactly, so only where places where false positives can be tolerated is where we're looking to do that. You don't want to take the humans out of the equation just yet or maybe ever. For some of the simple things we have and we can and we will, but some of the complicated things it's like just stop and look at it and think about it for 90 seconds and then make the action where to come up with how to program that 90 seconds of thought is like. Maybe talk about it. It'd be complete. Bounce it off of it this way, wait, wait, okay. Let me explain it to somebody a second time and make sure it's right and then go and do it quick. Just philosophically that's where I am. To get a machine to do that. So Jim, you're wearing the revolution award shirt. My understanding in Provada is now one of two time award winners. If I got it right, you're a commander award winner. Maybe you could explain what that means and what it means to you and your company. Sure, so the commander award is really about getting other folks in your organization using Splunk, either looking at a dashboard or to report the award digging down into the data. And so why I won the award was really around like our use of Docker containers. So it was really important to me that developers, people in DevOps, people in support, don't really have like a strong like network operations function, but those types of folks that they're all looking at the exact same thing, all the exact same tools, all the exact same data. So kind of as part of that mission it's just I hold trainings, I hold office hours, I've got one of my DevOps folks down here today or at the conference to then kind of spread the Splunk gospel, show people how to use it, if they've got questions, all that sort of stuff. And then the other part of that is really just showing people what we can do and advocating for the making decisions based on the data, we have it in data, have it in Splunk, let's look at that to make the decision. So that's really what that commander award's kind of all about. Doing the Docker stuff, you're a bit of a trailblazer. So we're only a few years into this container initiative. I was walking the show floor, I even saw some companies looking at like the serverless technology. What led you to kind of put these pieces together and tell us a little bit about kind of the community that you lean on to learn these things? Yeah, so the technology trend around containers was very strong and very fast, like with Amazons, especially like when they came out with their ECS orchestration, it was really fast and very strong and really the technology trend kind of led me into it. And then the developers being like, we're going to use Docker, we're going to have to figure, you're going to have to figure out how to Splunk it. So really from the very beginning, I've gone through each and every sort of possible way to get data out of a Docker container into Splunk. And part of that is networking with the Splunk folks, pretty good relationship with the fella that wrote the logging driver that went into the Docker open source project and like looked at the code reviews and all that. And then it's really just trying it out, trying things out and eventually kind of got to the sweet spot now where I've got the developers are all using local Docker compose and that's configured a certain way. Then when we run in Amazon, it's using Amazon ECS where I've also been working on Kubernetes for a while. And the way that you configure Docker in each of those environments is totally different. The code running in the container is exactly the same. So we've realized that vision, but the runtime environment is totally different. So kind of where we're at now, the config may be totally different on the logging drivers, but in the end, when you load up Splunk and you look at it and Splunk, it's exactly the same, whether it's your local laptop and Amazon in production, staging or whatever. And I think my kind of favorite part in terms of like the Splunk commander award in getting folks using Splunk is that the way that I have it set up now, there is literally no local log file for the developers on their laptop. It just doesn't exist. It all goes out to Splunk. So you can do a lot with grep and text pad and stuff on your local laptop and I get that, but now that they're in Splunk and it's just, it's been a great way to get folks on board with what it's going to look like in production. I know what it looks like in dev so I can make sure that my logs are good and I'm logging enough and not too much and all that stuff. So that's really where Docker is really, software is the same. Now we've got the logging the same, the tools are all the same, but then the runtime bits, those are a little bit different and that's abstracted away, hopefully. Jim, what does a DevOps guy want from a vendor? You got a lot of open source stuff that you're working with. You got a lot of different tooling. What do you look for in a vendor? What's the thumbs up and positives and what stuff really kind of ticks you off? Well, so we're a key trusted vendor for a lot of healthcare organizations so I can kind of talk about how we, if a customer or a user comes to us with a problem, doesn't matter what it is, it's our problem and we go through exhaustive lengths to identify where the problem actually is and so that may be in our code, that may be in another vendor's code, some third party, some open source thing, doesn't matter where after the evidence, where after the facts, we don't care if it's not in our code, we're going to help our customer be successful and that's what we would want from any vendor, right? So if we contact them with a support case, we've got a problem, we don't want any of this, ugh, looks like a firewall problem or something like get to the data, get to the facts and if you can prove, if the vendor can prove that the problem is somewhere else, great, but we want a reproducible test case, we want this whole finger pointing thing is like, it's horrible inside of an organization in terms of like running operational systems but then when you've got like Azure, Google Cloud, Amazon Cloud, Salesforce, ServiceNow, all these things all working together, like you can't, people just get to own the problem basically and that's what we do, right? So if the customer comes to us with an issue, it's our problem and then we go from there and figure it out and that's really what, any vendor that we work with, especially like a production operational sort of system, that's really what we look for. So you look for collaboration and focus on solving the problem, not the finger pointing, you know, a virtual single throat to choke, if you will. Yeah, exactly. Jim, well thanks very much for joining us on theCUBE, it was great to have you. Yeah, thank you, thank you very much. You're welcome, appreciate it. All right, keep it right there, everybody, Stu and I'll be back. Hashtag SplunkConf18, you're watching theCUBE, right back.