 From theCUBE Studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE and ETR. This is Breaking Analysis with Dave Vellante. The rise of Kubernetes came about through a combination of forces that were in hindsight, quite a long shot. Amazon's dominance created momentum for cloud native application development and the need for newer and simpler experiences beyond just easily spinning up compute as a service. This wave crashed into innovations from a startup named Docker and a reluctant competitor in Google that needed a way to change the game on Amazon and the cloud. Now add in the effort of Red Hat, which needed a new path beyond enterprise Linux. And oh by the way, it was just about to commit to a path of a Kubernetes alternative for OpenShift and figure out a governance structure to herd all the cats in the ecosystem and you get the remarkable ascendancy of Kubernetes. Hello and welcome to this week's Wikibon Cube Insights powered by ETR. In this Breaking Analysis, we tapped the backstories of a new documentary that explains the improbable events that led to the creation of Kubernetes. We'll share some new survey data from ETR and commentary from the many early innovators who came on theCUBE during the exciting period since the founding of Docker in 2013, which marked a new era in computing. Because we're talking about Kubernetes and developers today, the hoodie is on. There's a new two-part documentary that I just referenced, it's out and it was produced by Honeypot on Kubernetes, part one and part two, tells the story of how Kubernetes came to prominence and many of the players that made it happen. Now a lot of these players, including Tim Hawken, Kelsey Hightower, Craig McClucky, Joe Beta, Brian Grant, Solomon Hikes, Jerry Chan and others came on theCUBE during the formative years of containers going mainstream and the rise of Kubernetes. John Furrier and Stu Miniman were at the many shows we covered back then and they unpacked what was happening at the time. We'll share the commentary from the guests that they interviewed and try to add some context. Now let's start with the concept of developer-defined infrastructure, DDI. Jerry Chan was at VMware and he could see the trends that were evolving. He left VMware to become a venture capitalist at Greylock. Docker was his first investment and he saw the future this way. What happens is when you define infrastructure and software, you can program it, you make it portable and that's the beauty of this cloud wave, what I call a DDI is now to your point is every piece of infrastructure from storage, networking, to compute has an API, right? And AWS was an early trend where S3, EBS, EC2 had API. And I see- As building blocks too. As building blocks. Is it exactly? The Lego blocks- Not monolithic. The monolithic building blocks, every little blue and bone block has its own API and just like Docker really is the API for this, you know, the cloud enables developers to define how they want to build their applications, how to network them, you know, as Will's talked about and how you want to secure them, how you want to store them. And so the beauty of this generation is now developers are determining how apps are built, not just at the end user, you know, iPhone app layer, the data layer, the storage layer, the networking layer. So every single level is being disrupted by this concept of DDI and where, how you build, use and actually purchase IT has changed and you're seeing the incumbent ventures like Oracle, VMware, Microsoft try to react but you're seeing a whole new generation start. Now what Jerry was explaining is this new abstraction layer that was being built. Here's some ETR data that quantifies that and shows where we are today. The chart shows net score or spending momentum on the vertical axis and market share which represents the pervasiveness in the survey set. So as Jerry and the innovators who created Docker saw the cloud was becoming prominent. And you can see it still has spending velocity that's elevated above that 40% red line which is kind of a magic mark of momentum. And of course, it's very prominent on the X axis as well. And you see the low level infrastructure, the virtualization and that even floats above servers and storage and networking, right? Back in 2013, the conversation with VMware, and by the way, I remember having this conversation deeply at the time with Chad Sackage, was we're going to make this low level infrastructure invisible and we intend to make virtualization invisible, i.e. simplified. And so you see above the two arrows there related to containers, container orchestration and container platforms, which are abstraction layers and services above the underlying VMs and hardware. And you can see the momentum that they have right there with the cloud and AI and RPA. So you had these forces that Jerry described that were taking shape and this picture kind of summarizes how they came together to form Kubernetes. And the upper left, of course, you see AWS and we inserted a picture from a post we did right after the first reinvent in 2012. It was obvious to us at the time that the cloud gorilla was AWS and had all this momentum. Now, Solomon Hikes, the founder of Docker, you see there in the upper right, he saw the need to simplify the packaging of applications for cloud developers. Here's how he described it back in 2014 in the cube with John Furrier. Tainer is a unit of deployment, right? It's the format in which you package your application, all the files, all the executables, libraries, all the dependencies and one thing that you can move to any server and deploy in a repeatable way. So it's similar to how you would run an iOS app on an iPhone, for example. So Docker at the time was a 30 person company and it just changed its name from dot cloud. And back to the diagram, you have Google with a red question mark. So why would you need more than what Docker had created? Craig McClucky, who was a product manager at Google back then, explains the need for yet another abstraction. You create the strong separation between infrastructure operations and application operations. And so Docker has created a portable framework to take a basically a binary and run it anywhere, which is an amazing capability. But that's not enough. You also need to be able to manage that with a framework that can run anywhere. And so the union of Docker and Kubernetes provides this framework where you're completely abstracted from the underlying infrastructure. You could use VMware, you could use a red hat open stack deployment. You could run on another major cloud provider like. Now Google had this huge cloud infrastructure, but no commercial cloud business compete with AWS, at least not one that was taken seriously at the time. So it needed a way to change the game. And it had this thing called Google Borg, which is a container management system and scheduler. And Google looked at what was happening with virtualization and said, you know, we obviously could do better. Joe Beta, who was with Google at the time, explains their mindset going back to the beginning. I started up Google compute engine, VM as a service. And the odd thing to recognize is that nobody who had been at Google for a long time thought that there was anything to this VM stuff. Right, because Google had been on containers for so long that was their mindset. Borg was the way that stuff was actually deployed. So, you know, my boss at the time who's now a cloud era booted up a VM for the first time and anybody in the outside world be like, hey, that's really cool. And his response was like, well, now what? Right, you're sitting at a prompt, like that's not super interesting. How do I run my app? Right, which is that's what everybody's been struggling with with cloud is not, how do I get a VM off? How do I actually run my code? Okay, so Google never really did virtualization. They were looking at the market and said, okay, what can we do to make Google relevant in cloud? Here's Eric Brewer from Google, talking on theCUBE about Google's thought process at the time. One of the things about Google is it essentially makes no use of virtual machines internally. And that's because Google started in 1998, which is the same year that VMware started. It was kind of brought the modern virtual machine to bear. And so Google infrastructure tends to be built really on kind of classic UNIX processes and communication. And so scaling that up, you get a system that works a lot with just processes and containers. So kind of when I saw containers come along with Docker, we said, well, that's a good model for us. And we can take what we know internally, which was called Borg, a big scheduler. And we can turn that into Kubernetes and we'll open source it. And suddenly we have kind of a cloud version of Google that works the way we would like it to work. Now, Eric Brewer gave us the bumper sticker version of the story there. What he reveals in the documentary that I referenced earlier is that initially, Google was like, why would we open source our secret sauce to help competitors? So folks like Tim Hawken and Brian Grant who were on the original Kubernetes team went to management and pressed hard to convince them to bless open sourcing Kubernetes. Here's Hawken's explanation. When Docker landed, we saw the community building and building and building. I mean, that was a snowball of its own, right? And as it caught on, we realized we know what this is going to. We know once you embrace the Docker mindset that you very quickly need something to manage all of your Docker nodes once you get beyond two or three of them. And we know how to build that, right? We got a ton of experience here. Like we went to our leadership and said, please, this is going to happen with us or without us. And I think the world would be better if we helped. So the open source strategy became more compelling as they studied the problem because it gave Google a way to neutralize AWS's advantage. Because with containers, you could develop on AWS, for example, and then run the application anywhere like Google's cloud. So it not only gave developers a path off of AWS, if Google could develop a strong service on GCP, they could monetize that play. Now, let me focus your attention back to the diagram which shows this smiling Alex Pulvy from CoreOS which was acquired by Red Hat in 2018. And he saw the need to bring Linux into the cloud. I mean, after all, Linux was powering the internet. It was the OS for enterprise apps. And he saw the need to extend its path into the cloud. Now, here's how he described it at an OpenStack event in 2015. Similar to what happened with Linux, like, yes, there is still need for Linux and Windows and other OSes out there, but by and large on production, web infrastructure, it's all Linux now. And you were able to get onto one stack. And how were you able to do that? It was by having a truly open, consistent API and a commitment to not breaking APIs and so on that allowed Linux to really become ubiquitous in the data center. Yes, there are other OSes, right? But Linux by and large for production infrastructure, what is being used. And I think you'll see a similar phenomenon happen for this next level upwards, where you're treating the whole data center as a computer instead of treating one individual instance as just a computer. And that's the stuff that Kubernetes and Mesa and someone is doing. And I think there will be one that shakes out over time. And we believe that'll be Kubernetes. So, Alex saw the need for a dominant container orchestration platform and you heard them. They made the right bet. It would be Kubernetes. Now, Red Hat, Red Hat's been around since 1993. So it has a lot of on-prem. So it needed a future path to the cloud. So they rang up Google and said, Hey, what do you guys have going on in this space? And Google was kind of non-committal, but it did expose that they were thinking about doing something that was, you know, pre-Kubernetes, it was before it was called Kubernetes, but hey, we have this thing and we're thinking about open sourcing it. But Google's internal debates and some of the arm twisting from the engineers was taking too long. So Red Hat said, well, screw it. We got to move forward with OpenShift. So we'll do what Apple and Airbnb and Heroku are doing and we'll build on an alternative. And so they were ready to go with MeSos, which was very much more sophisticated than Kubernetes at the time and much more mature. But then Google, the last minute said, Hey, let's do this. So Clayton Coleman with Red Hat, he was an architect and he leaned in right away. He was one of the first outside committers outside of Google. But you still had these competing forces in the market and internally there were debates. Do we go with simplicity or do we go with system scale? And then Goldberg from Google explains why they focused first on simplicity and getting that right. We had to defend of why we are only supporting 100 nodes in the first release of Kubernetes. And they explained that they know how to build for scale. They've done that. They know how to do it. But realistically, most of users don't need large clusters. So why create this complexity? So Goldberg explains that rather than competing right away with, say, misos or Docker swarm, which were far more baked, they made the bet to keep it simple and go for adoption and ubiquity, which obviously turned out to be the right choice. But the last piece of the puzzle was governance. Now Google promised open source Kubernetes, but when it started to open up to contributors outside of Google, the code was still controlled by Google and developers had to sign Google paper that said Google could still do whatever it wanted. It could sub license, et cetera. So Google had to pass the baton to an independent entity and that's how CNCF was started. Kubernetes was its first project. And let's listen to Chris Anizek of the CNCF explain. CNCF is all about providing a neutral home for cloud native technology. And it's been about almost two years since our first board meeting. And the idea was there's a certain set technology out there that are essentially microservice based that live in containers that are essentially orchestrated by some process. That's essentially what we mean when we say cloud native. And CNCF was seeded with Kubernetes as its first project. And as we've seen over the last couple of years, Kubernetes has grown quite well. They have a large community, a diverse contribution, contributor base and have done kind of extremely well. They're one of actually the fastest, highest velocity open source projects out there. Okay, so this is how we got to where we are today. This ETR data shows container orchestration offerings. It's the same XY graph that we showed earlier and you can see where Kubernetes lands. Not we're standing, that Kubernetes is not a company but respondents, they know they're doing Kubernetes. They maybe don't know who's platform. And it's hard with the ETR taxonomy. It's a fuzzy and survey data because Kubernetes is increasingly becoming embedded into cloud platforms and IT pros. They may not even know which one specifically. And so the reason we've linked these two platforms, Kubernetes and Red Hat OpenShift is because OpenShift right now is a dominant revenue player in the space and is an increasingly popular pass layer. Yeah, you could download Kubernetes and do what you want with it but if you're really building enterprise apps you're going to need support and that's where OpenShift comes in. And there's not much data on this but we did find this chart from Omnia which shows the container software market, whatever that really is, and Red Hat has got 50% of it. This is revenue. And we know the muscle of IBM is behind OpenShift so it's really not hard to believe. Now we've got some other data points that show how Kubernetes is becoming less visible and more embedded under the hood if you will as this chart shows. This is data from CNCF's annual survey. They had 1,800 respondents here and the data showed that 79% of respondents use certified Kubernetes hosted platforms. Amazon Elastic Container Service for Kubernetes was the most prominent of 39% followed by Azure Kubernetes Service at 23% and Azure AKS Engine at 17% with Google's GKE Google Kubernetes Engine behind those three. Now, you have to ask, okay Google, Google's management initially they had concerns, well, why are we open sourcing such a key technology? And the premise was it would level the playing field and for sure it has, but you have to ask that, has it driven the monetization Google was after? And I would have to say no, it probably didn't but think about where Google would have been if it hadn't open sourced Kubernetes. How relevant would it be in the cloud discussion? Despite its distant third position behind AWS and Microsoft or even fourth, if you include Alibaba, without Kubernetes Google probably would be much less prominent or possibly even irrelevant in cloud, enterprise cloud. Okay, let's wrap up with some comments on the state of Kubernetes and maybe a thought or two about where we're headed. So look, no shocker Kubernetes for all its improbable beginnings has gone mainstream. In the past year or so, we're seeing much more maturity and support for stateful workloads and big ecosystem support with respect to better security and continued simplification, but it's still pretty complex. It's getting better, but it's not VMware level of maturity, for example, of course. Now adoption has always been strong for Kubernetes for cloud native companies who start with containers on day one, but we're seeing many more IT organizations adopting Kubernetes as it matures. It's interesting, Docker set out to be the operating system of the cloud and Kubernetes has really kind of become that. Docker desktop is where Docker's action really is. That's where Docker is thriving. It's sold off Docker Swarm to Mirantis. It's made some tweaks. Docker has made some tweaks to its licensing model to be able to continue to evolve its business. You hear more about that at DockerCon. And as we said years ago, we expected Kubernetes to become less visible. Stu Miniman and I talked about this in one of our predictions post and really become more embedded into other platforms. And that's exactly what's happening here, but it's still complicated. Remember, go back to the early and mid cycle of VMware, understanding things like application performance. You needed folks in lab coats to really remediate problems and dig in and peel the onion and scale the system. You know, in some ways you're seeing that dynamic repeated with Kubernetes, security performance scale, recovery, when something goes wrong, all are made more difficult by the rapid pace in which the ecosystem is evolving Kubernetes. But it's definitely headed in the right direction. So what's next for Kubernetes? We would expect further simplification and you're going to see more abstractions. We live in this world of almost perpetual abstractions. Now as Kubernetes improves support for multi-cluster, it will begin to treat those clusters as a unified group. So kind of abstracting multiple clusters and treating them as one to be managed together. And this is going to create a lot of ecosystem focus on scaling globally, okay? Once you do that, you're going to have to worry about latency and then you're going to have to keep pace with security as you expand the threat area. And then of course recovery. What happens when something goes wrong? More complexity, the harder it is to recover. And that's going to require new services to share resources across clusters. So look for that. You also should expect more automation. It's going to be driven by the host cloud providers. As Kubernetes supports more stateful applications and begins to extend its cluster management, cloud providers will inject as much automation as possible into the system. Now, and finally, as these capabilities mature, we would expect to see better support for data intensive workloads like AI and machine learning and inference. Scheduling with these workloads becomes harder because they're so resource intensive and performance management becomes more complex. So that's going to have to evolve. I mean, frankly, many of the things that Kubernetes team way back when, you know, they backburned early on they, for example, that you saw in Docker swarm or mesos, they're going to start to enter the scene now with Kubernetes as they start to sort of prioritize some of those more complex functions. Now, the last thing I'll ask you to think about is what's next beyond Kubernetes? You know, this isn't it, right? With serverless and IoT and the edge and new data heavy workloads, there's something that's going to disrupt Kubernetes. So in that, by the way, in that CNCF survey, nearly 40% of respondents were using serverless and that's going to keep growing. So how is that going to change the development model? You know, Andy Jassy once famously said that if they had to start over with Amazon retail, they'd start with serverless. So let's keep an eye on the horizon to see what's coming next. All right, that's it for now. I want to thank my colleague, Stephanie Chan, who helped research this week's topics and Alex Meyerson on the production team who also manages the Breaking Analysis podcast, Kristen Martin and Cheryl Knight helped get the word out on social. So thanks to all of you. Remember, these episodes are all available as podcasts wherever you listen, just search Breaking Analysis podcast. Don't forget to check out ETR's website at ETR.ai. We'll also publish a full report every week on wikibon.com and siliconangle.com. You can get in touch with me. Email me directly, david.volante at siliconangle.com or DM me at dvolante. You can comment on our LinkedIn posts. This is Dave Volante for theCUBE Insights, powered by ETR. Have a great week, everybody. Thanks for watching. Stay safe, be well, and we'll see you next time.