 And welcome back everybody, this is theCUBE, SiliconANGLE's flagship TV production. We are here live at .com 2012, Splunk's annual user conference. We're at the Cosmopolitan Hotel in Las Vegas. My name is Jeff Kelly, I'm from Wikibon.org and I'm joined by my co-host Jeff Frick from SiliconANGLE. Thank you Jeff, welcome back everyone. We're going wall-to-wall today again. I think we've had a lot of terrific guests. Again, we hope you'll jump on and join us on the journey, the data journey, hashtag data journey, jump on Twitter. Let us know what you think. So now we're happy to have another customer, which is always really fun to hear from customers how they're using the technology, how it's changed their business. So we'd like to introduce Bo Christensen from Ping Identity, welcome to theCUBE Bo. Thanks for having me. Glad to have you aboard. So before we jump in, for those people out there that don't know what Ping Identity is, want to give us a quick little overview over Ping Identity. Sure, Ping Identity does a single sign-on identity security for 800 of the world's largest companies worldwide. We've secure corporate identities of millions of employees around the world. We do mobile security, API security, and federated identity as well. Wow. So security just continues to be a hot topic here at the show and a big use of Splunk software to look for patterns and to do all kinds of things. So you're an active user. Can you tell us some of the things that you guys use Splunk for? Yeah, absolutely. So I'm in the on-demand operations group. We're part of a DevOps group, Site Reliability Engineer for that group. So we use it for operational intelligence, like most of the Splunk customers. We also use it for security. It's tied in directly to our security infrastructure. We also use it for business intelligence and our support personnel use it for diagnosing customer issues kind of in real time as well. Wow, so you've got it kind of spreading across a pretty broad swath of users. We do. We have multiple apps that are for the different groups and designed for the different groups. We also use it for all of our reporting. We're integrated directly with PagerDuty. So all the real-time messages and event types coming out of the system are piped directly to PagerDuty and come out to me. I think I'll call right now actually and my team as well. So we're seeing a common trend among adoption of customers is they download Splunk for free and they kind of get started in one use case usually in IT monitoring and infrastructure in the data center and then they expand throughout the organization. Does that model apply to you or did you have a different story? No, absolutely. It's the same for us. We downloaded it and we're going to use it originally to manage. We had hundreds of VMs across multiple data centers and had a problem of managing and auditing access to sensitive log files. So it started there. And from there we kind of discovered how much information we could pull out of this machine data in our first generation product and our second generation product we were able to then integrate a lot of the machine data in log files with Splunk and having Splunk in mind and our dashboards to be able to pull additional business metrics, event types, alerting everything that we could out of the system. Right. We even we talked a little bit before we came on and you were saying that it's really expanded beyond all the IT guys and the SPART technical guys into the business side of the house. So I'm always kind of curious, how did that happen? Kind of what was our breakthrough app? Was it a presentation of some executive that they said, wow, if you can do that, can you do this or somebody looking over for somebody's shoulder at lunch waiting to go to lunch and said, oh my gosh, what do you got going on there? Tell us a little bit about that. That happens too often now actually. It makes a lot of work for us. But yeah, so I think we originally built dashboards for my team and operations to see traffic and the amount of traffic per day and how it fluctuated during the week. And then we were then able to trend that data over time. And of course, like any good ops team have a weekly status report that goes out. And ours happens to go to the execs too. So we ended up starting to put some of that trending data into the executive reports. And then of course, more and more requests filter down about, oh, you can see that? Can you see this too? Can you tell me this about this customer and how are these connected? And what kind of more business intelligence can we get out of the system as well? And we eventually got sick of running customer reports and build them their own application then too. So they could just log in now and see for themselves who's active in the system, what changes are they making, should I call this guy, are these guys over there, user limit, that kind of thing. So you built them their own application then to go on top? Just so they'd stop bugging us. Awesome, that's great. I think that speaks to when even a CEO and an executive can use a tool. It's got to be pretty easy to do it. So just a little dig there at our CEO for us watching. I'm going to be careful. We both have ties on, Jeff. That's true. That's true. So you mentioned DevOps earlier. We've heard that a couple times yesterday actually during our broadcast. And so I'm wondering, what makes Flunk such a useful tool in that type of environment? We're seeing that more and more in this kind of agile, kind of lean production type of environment where you can't wait months to run something out of an update or a new application or whatever. And so that requires Ops and Dev to really work closely together. So talk a little bit about how you guys see DevOps. And specifically, how does Splunk fit into that? Because as I said a couple of times, Splunk seems to be in a name for that type of approach. Yeah, it's because you can share data so easily. So our interpretation, and there's a bunch of DevOps, is to actually integrate the engineering and the operations team together into scrum meetings, into planning and architecture. We have direct access to engineering resources, and they have direct access to operational needs through us. We're integrating the same ticketing system, GRO, all that stuff. And Splunk really enables us to give them access to our own operations dashboard. So we can say, hey guys, we just deployed a new cloud desktop four hours ago, and we saw an increased error rates. Here's what we're seeing. And they can have direct access to that data. They can see the stack traces. They can see untrapped errors coming through the system. If it's related to their piece of the code, they'll pretty much know right away and be able to tell us how to fix it or roll out a new fix for it in a really short period of time. So instead of us providing production access to developers in certain situations, it's most situations not a good idea. This is generally not a good idea. So Splunk allows them unfettered access to all the log files on production systems without giving them huge access to them. So just as an example, pick a processor or an incident. How many days would it have taken before? And how many people had to get touched along that in round numbers? And what's pick one now that you've got the systems in place that you have now? What does that look like? So I think prior to production deployment for us, we had a hack day inside Ping where we had all of our regional solutions architects use Ping One, our on-demand application for the first time and walk them through training. And the entire DevOps team was in the room with laptops. Which is how many people, tens of people, five people? No, our DevOps team is, yeah, at that point it was probably 15 people or so and then had the entire group of sales engineers basically in the room too. Not an inexpensive investment in human capital. Right, yeah, sure. But they were able to, the RSEs could say, you know, here's the error code that I just got. And we had engineers walking around and plugging that error code into Splunk and they were able to tell them right away, here's what you did wrong, or hey, we need to open a Jira ticket for that right away, right now. So that, just having our developers have access to, you know, Splunk as they're, you know, talking to sales engineers and the first actual users of the system was fantastic for us, I think. That's great. Yeah, can you contrast that to some of the tools you've probably used throughout your career and how we're really changing and how that ties into this whole big data movement and the whole idea of being more agile, being more user-friendly to the extent possible, to really be able to adapt quicker versus the kind of more static, more difficult to work with type of tools we've seen in the past? Yeah, absolutely. I mean, in the past, you know, you had a syslog search tool that you could use or you could have some guy logged into one machine and hope that everybody was hitting that machine. Splunk allows us to be far more distributed. We can have, you know, hundreds of nodes spread across multiple data center regions now, all in one place that you can see across all of them instead of, you know, bringing all that data down into a central logging server and then grepping stuff so you're blue in the face. Yeah. Great. Well, it sounds like it's been a terrific implementation. Absolutely. And a lot of ROI and are you seeing it now spun into being able to develop new stuff better, faster, and put those resources to work that may be before we're kind of chasing tales of things going so well? Absolutely. Our on-demand team operated with just two site reliability engineers for four years until the beginning of this year we added a third. So the operations team has been able to run really, really thin. And we've been able to get, you know, incredible alerts and tunely alerts and predictive alerts out of the system too. So yeah, we couldn't live without it at this point. And on the BIS site, on the executive reports, how far up have you gone? We've gone all the way up to the top. Oh, yeah. Absolutely. Take it a little as a Splunk report. Andre loves to see you. Awesome. That's great. Well, that's great. So, you know, we're here at obviously.conf. So there's over 1,000 attendees. And we've heard a lot about the importance of the community around Splunk and helping each other kind of find new use cases and maybe reuse applications that were developed in one organization for another. Just what's your take? I mean, how important is the community environment around Splunk to what you're doing and how much does that help? And, you know, are you giving back? How does it work? Yeah, it's the community and Splunk base and the information that we've been able to get off that. Our biggest win with Splunk was designing our own apps and building our own apps. So we couldn't have done that without the applications that were already up there. You know, we pulled down a couple of them and pulled apart how they worked and how the menus operated. And because all the stuff's pretty open, it's just XML. You can go in and pretty easily figure it all out and maybe steal some colors here and there. We don't tell that. Save as. We like save as. Right? And we've been able to give back to and publishing our own, our on-premise app is Pink Federate. And a lot of our Pink Federate customers are also Splunk customers and been asking us, you know, do you guys have an app for this? Do you have an app for this? So we were finally able to give back and publish an app for them up on Splunk base too, which has been great. We've got a lot of happy customers. Great. That's interesting because I think, you know, we follow the big data movement pretty closely. And one thing that we're seeing, we're not seeing as much uptick and it's kind of the applications, especially out of the box applications from the vendor community. We're starting to see them from vendors like Splunk. But I also think really it's gonna depend a lot on and users who are building their own applications because, you know, after all, you want to make use of this data to, you know, for competitive advantage. So you don't necessarily want a shrink wrapped app that everybody else's using. So there's, you know, there's certain elements of building applications in big data scenarios. It's a little different than a traditional scenario. Yep. And customers finding different ways to use the application. You know, I think Tom was saying earlier, somebody like tracking elevators with it. Right. Yeah, right. They never would have thought of. Yeah. It could go anywhere. I can't recall the earlier guests, but we were talking about, you know, all the new different types of equipment and machines that are being equipped with sensors that you can't even think of right now. But, you know, and who knows, your shoes, your watch, yeah. So, you know, in some ways it's a little bit, a little bit scary, but it's really exciting when you think about all the opportunity. Absolutely. Yeah, very cool. Well, great, well, thanks for stopping by. Again, Bo Christensen from Ping Identity, telling us again that if you are a Splunk practitioner in your company, you've got an opportunity to make some ways with the boss, show them what they can do, hopefully get a raise, build them a little app, get them off your seat and make them pay for lunch when they come and look over their shoulder and you have a cool app. And, yeah, take me out. So anyway, great story, Bo. Thanks for joining us on theCUBE. We really appreciate it. So, again, we've got a few more guests lined up. We've been going wall-to-wall here at the Splunk Conf 2012 and we will be back in a minute with our next guest.