 Live from Barcelona, Spain, it's theCUBE, covering KubeCon CloudNativeCon Europe 2019. Brought to you by Red Hat, the CloudNative Computing Foundation and ecosystem partners. Hi and welcome back. This is KubeCon CloudNativeCon 2019 here in Barcelona. 7,700 in attendance according to the CNCF Foundation. I'm Stu Minimanai, co-host for this week is Corey Quinn and happy to welcome back to the program a CUBE alum, Rob Somsky, who's the product manager for Red Hat OpenShift. Rob, thanks so much for joining us. Happy to be here. All right, so a couple of weeks ago, we had theCUBE in Boston, my short drive for me didn't have to take a flight as opposed to I'm doing okay with the jet lag here, but Red Hat Summit was there and it was a big crowd there and the topic we're going to talk about with you is operators and it was something we talked about a lot, something about the ecosystem, but let's start with there. For an audience that doesn't know what is an operator, how does that fit into this whole CloudNative space in this ecosystem? And where can you hire one? So there's software programs first of all and the idea of an operator is everything it takes to orchestrate one of these complex distributed applications, databases, messaging queues, machine learning services, they all are distinct components that all need to be life-cycled and so there's operational expertise around that and this is something that might have been in a bash script before, you had a Wiki page, it's up just in your head and so it's putting that into software so that you can stamp out many copies of that. So the operational expertise from the experts, so you want to go to the folks that make MongoDB for Mongo, for Redis, for Couchbase, for TensorFlow, whatever it is, those organizations can embed that expertise and then take your user configuration and turn that into Kubernetes. Okay and is there automation in that? When I hear the description, it reminds me a little bit of robotic process automation, or RPA, which you talk about how can I harrow them, RPA is, well, there are certain jobs that are rather repetitive and we can allow software to do that so maybe that's not where it is but help me put it into the... No, I think it is. Okay, awesome. Yeah, because if you think about it, there's like a certain amount of toil involved in operating anything and then there's just like mistakes that are made by humans when you're doing this and so you would rather just automate away that toil so you can spend your human capital on like higher level tasks so that's what operators are all about. All right, great. Do you find that operators are a decent approach to taking things that historically would not have been well suited for auto-scaling for example because there's manual work that has to happen whenever a node joins or leaves a swarm that is that something that operators tend to address more effectively or am I thinking about this slightly in the wrong direction? Yeah, so that you can do kind of any like Kubernetes event you can hook into so if your application cares about nodes coming and leaving for example, this is helpful for operators that are operating the infrastructure itself, which OpenShift has under the hood but you might care about when new namespaces are created or this pod goes away or whatever it is. You can kind of hook into everything there. So effectively becomes a story around running stateful things in what was originally designed for stateless containers. Yeah, that can help you because you care about nodes going away because your storage was on it for example or now I need to rebalance that. Whatever that type of thing is it's really critical for running the stateful workloads. Okay, maybe give us a little bit of context as to the scope of operators and any customer examples that you have that could help us get a little bit of concreteness to it. Yeah, so they're designed to run almost anything. So every common workload that you can think about on like an OpenShift cluster, you've got your messaging queues. So we have a product that uses an operator AMQ Streams, it's Kafka and we've got folks that heavily use the Prometheus operator. So I think there's a quote that's been shared around about one of our customer's ticket master and everybody needed some container native monitoring and everybody could just figure out Prometheus on their own or they could use the operator. And so they were running I think like 300 some instances of Prometheus and Dev and staging and this team, that team, this person's just screwing around with something over here. So instead of being experts in Prometheus they just use the operator and then they can scale out very quickly. Yeah, that's great because one of the challenges in this ecosystem, there's so many pieces of it. And we always ask it's like, okay, how many companies need to be expert on not just Kubernetes or but any of these pieces. So help us, how does this tie into the CNCF and all the various projects that are? Yeah, I think you've nailed it. I mean, you have to integrate all this stuff together and that's where the value of something like OpenShift comes at the infrastructure layer is you've got to pick all your networking and your storage and your DNS that you're going to use and wire all that together and then upgrade that life cycle it. And so the same thing happens at a higher level too. So you've got all these components getting your fluency pods down to operating things like Istio and service meshes, serverless workloads. All this stuff needs to be configured and it's all pretty complex and it's moving so fast nobody can be an expert. So the operator is actually the expert embedded from those teams, which is really awesome. You said something, I think before we got started a little bit about the end of a certification program for operators, what is that about? Yeah, so we think of it as a super set of our community operators. So we've got the TensorFlow community, for example, curates an operator, but for companies that want to go to market jointly with Red Hat, we have a certification program that takes any of their community content or some of their enterprise distributions and make sure that it's well tested on OpenShift and can be jointly supported by OpenShift and that partner. So if you come to Red Hat with a problem with a MongoDB operator, for example, we can jointly solve that problem with MongoDB and ultimately keep your workload up and keep it running. So we've got that times a bunch of databases and all kinds of services like that. And you can access those directly from OpenShift which is really exciting. So one click install of a production ready Mongo cluster. So you don't dig through a bunch of documentation for how that works. All right, so Rob, are all of these specific only to OpenShift or will they work with other flavors of Kubernetes? So most of the operators work just against a generic Kubernetes cluster. Some of them also do hook into OpenShift to use some of our specialized security primitives and things like that. And so that's where you get a little bit more value on OpenShift, but you know, you're just targeting Kubernetes at the end of the day. What are you seeing customers doing with this specifically? What, I guess, what user stories are you seeing that is, I guess, validating that this is the right direction to go in? Yeah, so you think it's a number of different buckets. The first one is seeing folks running just services internally. So you've traditionally had like a DBA team that maybe runs a shared database tier and folks are bringing that into the container native world from their VMs that they're used to and so using operators to help with that and so now it's self-service. So you have a dedicated cluster infrastructure team that runs clusters and then gives out quota and then you're just eating into that quota to run whatever workloads that you want in an operator format. So that's kind of one bucket of it and then you see folks that are building operators for internal applications. So they've got deep expertise in one team but if you're running any enterprise today, especially like a large-scale e-commerce shop, there's a number of different services. You've got caching tiers and load balancing tiers. You've got front-ends, you've got back-ends, you've got queues. So you can build operators around each one of those and so that those teams, even when they're sharing internally, hey, where's the latest version of your stack? Here's the operator, go to town, run it in, staging, QA, all that type of stuff and then lastly, you see these open source communities building operators which is really, really cool. So something like TensorFlow, the community curates an operator to get you one consistent install so everyone's not innovating on 30 different ways to install it and you're actually using it. You're doing high-level stuff with TensorFlow. Yeah, it's interesting as you laid out. Some of these, okay, well, a company is doing that because it's behind something, others you're saying it's a community. Remind me, Red Hat's long history of helping to give, if you will, adult supervision for all of these changes that are happening in the world out there. Yeah, I mean, it's a fast-moving landscape and some tools that we have like our operator SDK are helping to tame some of that so you can get quickly up and running building an operator whether you are one of those communities, you are a commercial vendor, you're one of our partners, you're one of our customers. We've kind of got tools for everybody. Yeah, anything specific in the database world? It's something we're seeing that Cambrian explosion in the database world? Yeah, I think, I mean, folks are now finally wrapping their heads around that Kubernetes is for all workloads and to make people feel really good about that, you need something like an operator that's got this extremely well-tested code path for, okay, what happens when these databases do fail? How do I fail it over? And it wasn't just some person that went in and made this, it's the expert, it's the folks that are committing to MongoDB, to CouchBase, to MySQL, to Postgres. So that's a really exciting thing is you're getting that expertise kind of as an extension of your operations team. All right, for people here at the show, are there sessions about operators or what's the general discussion here at the show for your team? Yeah, there's a ton, even too many to mention. There's from a bunch of different partners and communities that are curating operators, talking about best practices for managing upgrades of them, users, all that kind of stuff. I'm going to be giving a keynote, kind of an update of some of the stuff that we've been talking about here later on this evening. So yeah, it's kind of all over the place. What do you think right now in the ecosystem is being most misunderstood about operators, if anything? I think nothing is quite misunderstood. It's just wrapping your head around what it means to operate applications in this manner. And so just like Kubernetes components, there's this desired state loop that's in there. And you need to wrap your head around exactly what needs to be in that. Your declarative state is just the Kubernetes API, so you can look at desired and actual and make that happen, just like all the Kube components. And so it's just like a different way of thinking. And we had a panel yesterday at the OpenShift Commons about operators. And one of the questions that had some really interesting answers was, what did you understand about your software by building an operator? Because sometimes you need to tease apart some of these things. I had hard-coded configuration here. One group shared that their leader election was actually not working correctly in every single instance, and their operator kind of forced them to dig into that and figure out why. And so I think it's a give and take that's pretty interesting when you're building one of these things. Do you find that customers are starting to rely on operators to effectively run their own, for example, MongoDB inside of their Kubernetes clusters, rather than depending upon a managed service offering provided by their public cloud vendor, for example. Are you starting to see people just stop, effectively start reducing public cloud to baseline primitives and a place to run containers rather than the higher level services that are starting to move up the stack? Yeah, so a number of different reasons for that too. So you see this for, you know, those services, if you find like a bug in that service, for example, you're just kind of out of luck. You can't go introspect the versions, you can't see how those components are interacting. And so with an operator, you have an open-source stack. It's running on your cluster and your infrastructure, and so you can go introspect exactly what's going on. You know, the operator has that expertise built in, so it's not like you can screw around with everything, but you have a much more insight into what's going on. Another thing you can't get with like a cloud service is you can't run it locally. So if you've got developers that are, you know, doing development in an airplane or just want to have something local, so it's running fast, you can, you know, put your whole operator stack right on your laptop. You know, not something you can do with a hosted service, which is really cool. And, you know, most of these are open-source too, so you can go see exactly how the operator is built. It's very transparent, especially if you're going to trust this for a core part of the infrastructure, you really want to know what's going on under the hood. Okay, just to double-check, you know, since all of this can run on OpenShift, it is, you know, agnostic to where it lives, whether it's a public cloud or your data center. So these are truly hybrid services. So if you're migrating your database here, for example, over, now you have a truly hybrid, just targeting Kubernetes environment, so you can move that in any infrastructure that you'd like. And this is one of the things that we see OpenShift customers do, is some of them want to be cloud to cloud, cloud to on-prem, you know, different environments on-prem only, because you've got database workloads that might not be leaving or, you know, you've got a mainframe that you need to tie into a lot of our FSI customers. And so operators can help you there where you can't move some of those workloads. Yeah, cloud on-prem makes a fair bit of sense to me. One thing I'm not seeing as much of in the ecosystem has been cloud to cloud. What are you seeing that's driving that? I think everybody has their own cloud that they prefer for whatever reasons. And I think it's typically not even cost, it's tooling and cultural change. And so you kind of invest in one of those. And, you know, you, I think people are investing in technologies that might allow them to leave in the future. And, you know, operators and Kubernetes are leaving one of those important things. But that doesn't mean that they're not perfectly happy, you know, running on one cloud versus the other, running Kubernetes on top of that. Yeah. All right, well, Rob, really appreciate all the updates on operators. Thanks so much for joining us again. Absolutely, it's been a good luck. Good luck on the keynote. Thank you. All right, for Corey Quinn, I'm Stu Miniman. Back with more coverage, two days live, wall-to-wall, here at KubeCon CloudNativeCon 2019 in Barcelona, Spain. Thanks for watching.