 Live from San Diego, California, it's theCUBE. Covering KubeCon and CloudNativeCon, brought to you by Red Hat, the CloudNative Computing Foundation and its ecosystem partners. Welcome back to KubeCon, CloudNativeCon here in San Diego. I'm Stu Minim, my co-host is Justin Warren and one of the things we always love to do is really dig into some of the customer use cases and joining us to do that, Andre Ribca, who's the head of Compute Architecture in the CTO office at Bloomberg. Andre, thanks so much for joining us. All right, so just to set the stage, last year we had your colleague, Steven Bauer, came, talked about, your company's been using Kubernetes for a number of years. You're a member of the CNCF as one of those end users there and you're even an award winner. So congratulations on all the processes. You've been doing it for years, so all the problems I'm sure are already solved. So now we just have a big party, right? Yes. Well, I mean, certainly we are at this stage where things are quite mature and there's a lot of workloads that are running in Kubernetes. We run Kubernetes on premises. Steven has an excellent data science platform that does machine learning with GPUs and bare metal. We also have a really excellent team that runs basically platform as a service, generic platform as a service. Not GPUs, but effectively runs any kind of sales app or service and that's been extremely successful and there's a lot of interest in that and we also run Kubernetes on public cloud. So a lot of workloads for like Bloomberg.com actually are backed now by Kubernetes. Yeah, so we want to spend a bunch of time talking about the applications, the data, the services, you've built some passes there, but yeah, step us back for a second if you would and give us what led to Kubernetes and as you said, you've got your on-premises environment, you've got public cloud. Where was that when you started and what's the role of Kubernetes in that today? Sure, we started back in 2015 evaluating all kinds of sort of container orchestration platforms. It's very clear that developers love containers for its portability and you know, just ability to have the same environment that runs kind of on-premises or on your laptop and then it runs on the actual deployment environment, the same thing, right? So we looked at Mezos, Marathon, Cloud Foundry, even OpenShift before it was Kubernetes and we were in the office of CTO, we continuously sort of evaluate all different options and then once we make a decision, we recommend to the engineering team and work in partnership with engineers. So all of those awards and Everton actually, I want to say that this is really a kudos to our engineering team. We're just a small part of the puzzle. Now, as far as how we made the Kubernetes selection, it was a bit risky. We started with pre-Alpha version and I've read the Borg paper, how Google actually did Borg and when I sort of realized, well, they're trying to do the same thing with Kubernetes, it was very clear this is kind of, we're going to build on mature experience, right? So somewhat it was risky, but it also is safe bet because there was some good computer science and engineering behind the product. So we started Alpha version. The consumer web groups actually were one of the first deployments of the Kubernetes and they presented in the first KubeCon. It was an excellent talk on how we did Kubernetes and we came a long way since then. We've got sort of now probably about 8,200 clusters running and they run full high availability, DR minus one. I would say it's one of the most reliable environments that we have. We have frequently infrastructure auditors, hypervisors, the obviously hardware fails, which is normal and we rarely see any issues and actually no like any major issues whatsoever. So the things that we expected out of Kubernetes, things like reliability, elastic infrastructure, auto-scaling, the multi-tenancy, it all worked out. Higher density of sort of packing the nodes, that's another great sort of value add that we expected, but now we finally realize in that. So one question I've had from a lot of customers, particularly traditional enterprises who are used to doing things and they have a lot of virtual machine infrastructure. They're looking at Kubernetes but they're finding it somewhat opaque, a little bit scary. Talk us through how did you convince the business that this was the choice that we should make and that we need to change the way that we're developing applications and deploying applications and we want to do this with Kubernetes. How did you convince them that this was going to be, it was going to be okay in the end? Yes, yes, that's a really good question. A lot of people were scared and they were, is this going to break things or is this just a shiny new thing? And there was a lot of education that had to occur. We've shown a lot of POCs. Now the way we exposed Kubernetes was not just like a raw Kubernetes. We actually wanted to keep it safe. So we sort of stayed away from some like more alpha type of workloads and move towards kind of like the most stable things. And so we exposed this platform as a service. So the developers did not actually get to necessarily like Qubectl, you know, apply a config and just deploy the app. Like we actually had a really good sort of offering where we had kind of almost like a git flow kind of environment where you have your source control then you have CI CD pipeline. And then once it goes through all the checks and balances you deploy your containers. So from that perspective, we actually hid quite a bit of things that made things a bit dangerous or potentially a little more complicated. And that proven to be the right strategy because right now, as far as like the real ability, I would say this is probably one of the most reliable environments that we have. And this is by design. We basically tell the developers by default you're supposed to run at least two replicas, at least two data centers by default or two regions or two availability zones. And you can't change that. There's some people who are asking me like, can I just deploy just in one dataset? I'm not sorry, no. Like that by default is like that. And auto scaling on. So if one data center goes and you need the R minus one. So if you started with two minimum replicas then it auto scales to four or whatever that would be set. So I think we basically put a prototype of proof of concept relatively fast. And we got with the initial platform as a service from zero to actual delivery in about three months. A lot of building blocks were there and we just put it kind of the pieces of the puzzle together. That does echo a lot of the discussion that was had in the keynote today even was about looking at making Kubernetes easier to consume. Essentially by having all of these sensible defaults. Like you mentioned, you will have two replicas. It will run in these two different zones and kind of removing some of that responsibility for those decisions from the developers. How does that line up with the idea of DevOps which seems to be partly about making the developers a bit more responsible for their service and how it runs in production. But it sounds like you've actually taken a lot of that effort away from them by we've done all of this work for you so you don't have to think about that anymore. I mean a little bit of background. We have about 5,500 engineers. Expecting everybody to learn DevOps and Kubernetes is not realistic, right? And most developers really want to write applications and services that add business value, right? Nobody wants to really manage networking at the low level and there's a lot of still complexity in this environment, right? So as far as DevOps, we built shared kind of teams that have basically like a think of like centralize the SRE teams that build the core platform components. We have a world-class kind of software infrastructure group which builds those type of components. On top of the sort of the technology infrastructure team that caters to the hardware and the virtualization infrastructure built on OpenStack. So there's very much kind of a lot of common services plus shared services teams that built that as a platform to developers and that's how we can scale because it's very hard to do that if every team is just sort of replicating each one of those things. All right, so Andre, let's talk a little bit about your application portfolio. Bloomberg must have thousands of applications out there. From what you were describing, is this only for kind of net new applications? If I want to use it, I have to build something new replacing something else or can you walk us through kind of what percentage is on this platform today and how's that migration or transition? It's not necessarily net new. We actually did port quite a bit of the sort of classic Bloomberg services that developers expect to the platform and it's seamless to the developers. So we've been doing quite a bit of sort of Linux migration, meaning from like things like Solaris, AIX, and this platform was built purposefully to help developers to migrate their services. Now they're not sort of lift and shift type of migrations. You can't just expect the classic C++ shared memory app suddenly like jump and start being in containers, right? So there are some architectural changes, differences that had to be done. The type of applications that we see, they're just sort of microservices oriented. Bloomberg has been around since 1981 and they've been doing service oriented architecture since like early 90s. So things were already kind of in services kind of framework, kind of mentality and before we had service matches, Bloomberg had its own kind of paradigm of service matches. So all we're doing is kind of retrofitting the same concepts with the new frameworks and what we did is we brought in sort of like a new mentality of open source first. So most new systems that we built, we look for kind of what about if, we look for open source components that can fit in this particular problem set. So the applications that we have right now, we have quite a bit of data services, data transformation, pipelines, machine learning. There's quite a bit of the machine learning as far as like the actual learning part of training and then there is the inference part that runs quite a bit. We have quite a few of content services, like I mentioned, Bloomberg.com and many sort of things that you would normally think of like the content delivery services that run on Kubernetes. And I mean, at this point, we certainly try to be a little bit conscious about stateful services. So we don't run as much of databases and things like that. Eventually we'll get there once we prove the reliability and resiliency around the stateful sets and Kubernetes. Yeah, do you have an estimate internal or goals as to what percentage of applications are on this platform now and a roadmap going forward? I mean, it's hard to say, but going forward, I see majority of our services migrating to Kubernetes because for us, Kubernetes has become essentially a standardized compute fabric. One thing that we've been missing, there's a lot of open source projects delivered virtualized infrastructure, but that's not quite enough, right? You need other sort of concepts to be there and Kubernetes did deliver that for us. And more importantly, it also delivered us kind of almost like a multi-cloud strategy, kind of accidentally because none of the cloud providers have any standard APIs of any source, right? So even if you use Terraform, that's not necessarily multi-cloud. It's just like, you got to write HCO for each cloud provider. In Kubernetes, more or less, that becomes kind of a really solved problem. Well, so what flavor Kubernetes are you using? Do you leverage any of the services from the public clouds on Kubernetes? Yeah, I'm in excellent question. So we want to leverage managed offering as much as possible because things like patching the security of CVEs and things like this, I want somebody to take care of that for me and harden things and out of the box. So the key to our multi-cloud strategy is use managed offering, but based on open source software. So if you want to deploy services, deploy them on Kubernetes as much as possible, if you want to use databases, use managed database, but based on the open source software like Postgres or MySQL, and that makes it portable, right? To an extent, there's going to be some slight differences, but I do believe that managed is better than if I'm going to go and bootstrap VMs and manage my own control plan and the workers and things like that. And it isn't a lot of additional work that I think organizations generally did try to roll their own and do everything themselves. There's a lot more understanding, since the advent of cloud, essentially, that actually making someone else do this for what is essentially the undifferentiated heavy lifting, if you can get someone else to do that for you, it's a much better experience. Which is actually what you've built with the Kubernetes service for your developers is you are becoming that managed service for your app developers. I think a few enterprise organizations have tried to do that a little bit with centralized IT. They haven't quite got that service mentality there where I'm the product owner and I need to create something which my developers find is valuable to use so that they want to use it. This is exactly spot on. When I drove Malmberg six years ago, one of the things we wanted to do is effectively offer a public cloud services on-premises and now we are there. We actually have a lot of managed offerings, whether you want Kafka as a service, Queue as a service, or Cache as a service, or even Kubernetes, but not necessarily we want to expose Kubernetes as a service, we want to expose Platform as a service. So you hit the nail on the head because effectively developers, one kind of the same things that they see in the public cloud. Oh, I want fact function as a service. I want Lambda, something like this. Well, that's the type of platform as a service, right? So you're spot on. So, Andre, last question I have for you. You talked about the maturity of the managed offerings there. Something we've seen a lot this year is the companies that how am I going to manage across various environments there? You saw Microsoft with Azure Arc, VMware with Tanzu. What do you think of that? Is that something that interests you or anything else in the ecosystem that you still think needs to mature to help your business? Sure, sure. I mean, I think the use cases that trying to address are definitely near and dear to my heart because we are trying to be multi-cloud. And in order to be truly mature multi-cloud sort of company, we need to have sort of mature kind of multi-cloud control plane that has kind of the deployment address, CACD pipeline address, then it needs to address the security, not just like day one, but day two, alerting, monitoring, all of like, if I was just to have three different portals to look at, it's very complicated. You're going to miss things. I want one pane of glass, right? So what this company is addressing is extremely important and I see a lot of value in it. Now from my point of view and in general, what we prefer if it was an open source project that we could contribute and we could collaborate on, we'll still want to pay money for the support and what not. We don't want to just be free writers, right? But if it's an open source product and we can be part of it, it's not just read-only open source, that is definitely something that I will be very much interested in participating. And majority of the developers that we have, they're very happy to participate in open source. I think you've seen some of our contributors here. We have some people contributing to Kubeflow. There's many other projects. We have quite a bit of cool projects like the KS engineering with a powerful seal. If somebody wants to check it out, we've got some really interesting things. Well, Andre, really appreciate you sharing what you and your engineering teams are doing and thank you for all the contributions back to the community. For Justin Warren, I'm Stu Miniman, back with more of our three-day wall-to-wall coverage here, KubeCon, CloudNativeCon. Thank you for watching the Kube.