 Good afternoon, everyone. I'm Dr. Xiuxiao Gao from Stark and Wynne. You can call me Dr. Gao or XJ. Dr. XJ, yeah, if we are friends, that's okay. And I'm Mike Ferris also from Stark and Wynne. Thank you all for coming to our session. Let's jump right into the QA. Usually we have a QA at the end. You may wonder, are you crazy? Why are you doing a QA at the beginning? By the way, that's supposed to be a joke. I take a courtesy laugh if it's not funny. So, thank you. So we kind of do this presentation is like a question, ask and answering style. So that's why I did that joke there. I guess it's funny when I explain my joke also, right? Anyways, the first question is why cloud foundry application run time? In short, CFAR, actually quite often we just say cloud foundry, but we know CFAR application run time, CFDR, the same thing. Why is it so popular? What are the top three reasons you think CF is so popular? I see some thinking face in the audience, thinking for engaging in the presentation. So there are the top three reasons jumping into my mind, which are developer experience, developer experience, developer experience. So why I say that? So CF with this CFAR developers can just focus on code, focus on create your fancy apps. You just run CF push, you don't need to worry about how to build your code, how to run your code. You don't need to worry about how to scale it. Then for services, you also can just run CF create service, band service, easy, right? Because CF are doing all the hard work behind it to make this service work with your app. So the simpler, the better. So the next question will become, in order to get such good developer experience, how are we going to deploy and live with this CFAR? I bet you already know the answer. Now it's cloud foundry Bosch. Bosch is a super powerful tool for release engineering, deployment, and lifecycle management for either small scale or large scale distributed system. It has lots of features, such as self-healing. Like if some VMs goes down, Bosch will try to recreate it, recover it. Also like you can deploy CF onto different availability zones to achieve HAA. Also easy to scale, easy to work with different IS. However, there is always a bot or however, if you would like to manage multiple CFs environment, not only that, also system around it. Like you want to backup and restore your data, you want to monitor in what's going on, you want to manage your credentials. You want to manage your credentials, right? All those, then only just Bosch is not enough. We need something built around it, make those experience easier for you. I see your face said, oh, I agree. So I recommend a talk, which is from Charlie Baum, not from Charlie Baum, sorry, I have a migrant today. Charlie Baum from Comcast, he's going to share with you how in their company use a tool to deploy and manage this multiple production ready CF environment in a very efficient way and bring their business value. I see someone is taking picture of this, that means you buy it, thank you. So it's tomorrow, you see that right before lunch, you might believe it's a very good appetizer before your lunchtime. So make sure you go to that talk. So we already have this CF, it's so nice bringing you good developer experience. Why do we still need the Kubernetes? So this CFAR is entirely focused on the stateless, co-ordinated of so-called 12 factor applications. How about the case that developers need or would like to pack everything in a container? So in that way, the developer can run this app in a consistent way everywhere. It's the same on your laptop, it's the same in your dev environment, it's the same in your production environment. So it's always to me the top one reason for Kubernetes or any other container orchestration system is run containerized workloads. So Kubernetes kind of become a standard for container orchestration. It supports both stateful and the stateless workload and also give you more granular control. That can be a good or bad thing, right? If you want simple, how you want more control depends on. Then the next question will be great, Kubernetes will manage your container, take care for you. But who take care of your Kubernetes, right? So let's say if some of your app goes down, Kubernetes is very good. I'm going to put your container onto a different node. But how about the control plane? Who makes sure the control plane is running all the time, all that? I guess you already see one of the solution. The answer here is CFCR, Cloud Foundry Container Runtime. Actually it was called Kubo. Basically use that powerful Bosch I mentioned earlier to deploy your Kubernetes cluster. So then you have the feature like we mentioned earlier, right? Self-healing, easy scaling, HAA, all that. But as we pointed out earlier, if you want to manage multiple clusters, the system around it is needed to make it easier. One of the solution in the community is this gardener from SAP. It focuses on 100% of Kubernetes, there's no Bosch. The goal is provide a gardener as a service, basically use Kubernetes to deploy more Kubernetes. So if your team has lots of expertise on Kubernetes, I can see this is a very neat solution. So go into a little bit more detail. The basic idea you can see here, you will have a dedicated Kubernetes cluster, which are running a thing called the gardener. What it is, it's an extension API server with a bundle of customer controllers. So that will enable you to deploy your seed cluster and the shoot clusters. Then you'll ask, what is this seed cluster? So this seed cluster is deploying the control plans for your shoot clusters. Then the shoot cluster are the real place running your workload. So that way you can see the Kubernetes here will take care of the HAA, and they manage your control plan of your shoot cluster. They suggest to do one seed cluster per IIS per reason, like we showed in this diagram, because the network latency and other reasons here. Cool. So we talked about this different solution, CFAR, CFCR. The next question you may have, this is not an A or B question, right? Why not both? They have their own strengths, their own values and the purpose. So a user case, for example, their clients or companies, they use CFAR to run their cloud-native IPS. So they take advantage of that good developer experience. Then they run their services on Kubernetes to provide on-demand service for those CFIPS. So in that user case, there's a choice to make the experience better. One of them are Kibosh. So that enables you to run a CF command, say, CF create a service. Then it will just create a service out of a ham chart into the marketplace in CF. Certainly you can also do CF update service, CF band service, all that. One of the issues is when your apps in your CF world want to consume service from Kubernetes cluster, it's not that straightforward. Usually you have to deal with how they talk, right? You may have to use like a node part service. You may have to use a load balancer or ingress controller or all that. This blog post I put here is, I think it's a very cool idea and awesome work from my coworkers. So what this is, this is try to make a container to container direct networking between your application and CF application and services. So check it out if you want more details. So what about if your team are very familiar with Kubernetes but would like to use CF? Don't worry, you're covered because there is a project cloud foundry CF containerization. The basic idea is instead of using Bosch release for deploy CF, it is converted to the image so you can easily deploy your CF on your Kubernetes. There is also other solution based on this idea. Suji has SCF, IBM has CFE, was called cloud foundry enterprise edition before and now called cloud foundry enterprise environment. I know you are trying hard to read. What are those that mean? If you want to learn more, you can go to check their website or GitHub for more details. And also there is a talk today, later before dinner time. It's CF101 for Kubernetes users. So if you are really from Kubernetes world but want to explore more on CF, this is your talk, I recommend you guys go. So the last thing I would like to point out is with this last solution, we talk about the containerized CFAR solution. You will have nice to the containers, right? For example, you push your IPE, then you will have it running in Kubernetes. Then you see the container actually is the container for your Diego cell containers. Then you go inside again, that's real one running your IPE. So just keep in mind if you are adopting this solution. Next, it's time to give the mic to Mike. Laugh please. Hey guys, thank you XJ. So yeah, like XJ just talked about, we're getting into some different solutions that involve mixing cloud foundry in Kubernetes. And in this case that she actually just talked about, we are putting cloud foundry directly on Kubernetes via the CF containerization project. And so that is literally just taking every VM within cloud foundry, turning it into a container and running it on Kubernetes. Now, as she mentioned, you have, anybody who knows cloud foundry knows that apps run as containers within Diego cell VMs in the typical architecture. But in this architecture, those Diego cell VMs are now themselves containers. So you're running apps as containers within containers. And that can be a problem. Number one with debugging, if you get a problem, if you get, let's say you get a networking problem in an app, do you start debugging that? Which container level do you start debugging that at? You can also run into security issues between the discrepancies between the security settings of the two containers. Anyway, it can be a problem. And so there's incentive to move towards having the CF core components such as the cloud controller, go routers, basically everything except the Diego cells that actually run the apps, have those be containers running on Kubernetes and then have the actual apps themselves also be containers running on Kubernetes. And so that brings us to a project called Irini, which allows us to do just that. It effectively swaps out the Diego cell orchestration layer and allows you to use Kubernetes in its place so that, like I said, you can get an architecture like this where is there a mouse here? Yeah. So here's your Kubernetes, excuse me, your cloud foundry core components running as containers on Kubernetes. And then instead of the Diego cells running as Kubernetes and then apps as extra containers on top of those, we've gotten rid of Diego and Irini is the solution that allows you to just directly run your apps on Kubernetes after a CF push. So you're getting the same you're getting the cloud foundry developer experience out of this like XJ talked about, which is really the reason people use cloud foundry that CF push abstraction layer that lets developers not have to worry about the containers. But then under the hood, you're also you're running them on Kubernetes. And if you're a organization that has a lot of Kubernetes experience, you're now making you can effectively make use of that Kubernetes experience. So this is something that obviously there's incentive to do because people have put so much work into the Irini project. Problem with this is that these currently the CF core components, they are certified to run as Bosch deployed VMs that is the way to run them right now. And that is how so many organizations have scaled these up and found problems with them and then subsequently solve those problems and given those back to the community. So it's really a certified prod tested battle, battle hardened solution to deploying those CF core components, whereas deploying them as Kubernetes containers right now containers on top of Kubernetes. There and there will end up being a lot of unknown unknowns due to the lack of prod experience at bigger enterprises running that way. So instead a solution that fixes that is to do a blend of the of a couple of the solutions that we've talked to today. So you'll see here we are running the CF AR core, those core components that I talked about like the cloud controller, the go routers, basically everything except the Diego cells that are running your apps. Run those as the Bosch deployed VMs as it's tested as it's certified by the community and then use Irini to switch out the Diego layer for Kubernetes. And so you're still making use of your organization's Kubernetes experience. And then obviously the trade off here is that you need to you in this in this case you do your organization does need to have Bosch expertise in order to properly make use of that. Yeah, so those are a couple of the different ways that your organization can mend together Kubernetes and cloud foundry and kind of use them in tandem. And there's no best real best solution that comes that comes out of these it's really going to depend on the on your organization and details of them. So I'll run through a quick recap of all the options we've talked about. So there's cloud foundry, which is best deployed via Bosch via the CF AR cloud foundry application runtime release. There is Kubernetes, which is best deployed by the via Bosch via the cloud foundry container runtime deployment. There is the projects like the CF containerization project, which makes use of SUSE CF and IBM's CF EE project to take every single cloud foundry VM turn it into a container and toss it on Kubernetes. And then there is the arena project which removes the double nesting of containers and allows you to just directly run your containers on Kubernetes instead of running Diego on Kubernetes and your apps on Diego. And so the and then there and then there is taking the CF core components off of Kubernetes and instead running those as Bosch VMs so that you can know that you are going to scale properly and not hit. There are unknown unknowns with scaling that up into oblivion. And so which one to choose? The best answer is it depends. Like I said, it's going to depend on a couple of things. It's going to depend on your, I got a big, I meant to split this up, but so it's going to depend on the workloads you want to run like XJ talked about or Dr. Guy talked about do you want to get abstract abstract containerization completely away and just have your developers pushing straight source code? That's going to be where cloud foundry strengths are. Do you want to run containerized workloads where your developers are customizing those containers and they are taking responsibility for that containerization? Then you're going to use Kubernetes and so which workloads you want to run? What expertise do you have as an organization? If you have no Bosch experience, solutions like pushing your entire cloud foundry as containers directly to Kubernetes without doing any Bosch. That's going to be enticing for you, especially for the, for the case where you're an organization that is just getting into cloud foundry and you want to get your developers familiar with it and start hit the ground running. Meanwhile, you have your other operators learning Bosch and learning how to do this style of deployment where you have your Bosch deployed container, Bosch deployed cloud foundry core components. You know, yeah, currently yes, only because large enterprises and the cloud foundry community as a whole hasn't done that so far, but they're in the future that will very likely change as more enterprises start doing that. And cloud foundry running cloud foundry core components running as containers becomes a more battle hardened solution and the unknown unknowns with scaling that up become solved. Uh huh. Right. Uh huh. Right. Uh huh. Uh huh. Shots fired. Right. I agree. And yeah, in the future, that definitely will be the ideal once it's feted by the community. Yeah. Yeah. In the, yeah, I should, uh, thanks. Um, yeah. So like Dr. Nick mentioned in the future, obviously the once that, uh, solution of running your CF core components on directly on Kubernetes without having to Bosch deploy anything that will obviously be the ideal once it's a battle hardened by a lot of enterprises using it. Mm hmm. Guy in the red shirt. Get at him. Australian accent. Yeah. So any, the, uh, the three things that factor into which one of you, which one of these you should choose? What workloads you want to run? Which expertise does your organization have? And how far do you need to scale? And like Dr. Nick mentioned, that how far do you need to scale? That will change once, uh, CF core components as containerized components on Kubernetes rather than Bosch deployed VMs. Once that's more battle hardened, uh, that will be a better idea for companies to decide to run their business on at scale. Whereas right now that might not be the best idea because you might hit some issues that the community hasn't hit yet. And you might be ahead of the pack and trying to figure it out for yourself rather than relying on the community and their expertise. Um, and so with that, we'll bring on any other questions that anybody else has. If yes, if non 12 factor apps are more heavily supported by Kubernetes than yeah. Right. Right. That's a good point. Ideally not. If you have your CICD designed effectively enough to abstract that all the way. Yeah. From the developer's experience, from the developer's perspective, right. You're right. It becomes abstracted away and it does become a trade off between the expertise of the, uh, operators within the organization. So yeah. Good point. Thank you. Mm hmm. I get really sweaty. Mm hmm. Right. Do we have a question in the back? Yeah. Yeah. So I was just coming at it from the perspective of. Will cloud foundry work and should you trust it at scale running your business on a solution that is not the like the TM solution by the cloud foundry community. But yeah, if you if your organization has a. Incentive or if you want to decide, yeah, decide whether you want to run containers or VMs, then that's going to play in obviously too. Any other questions? Yeah. Right. Right. Ideally, the developer wouldn't be able to tell. And, um, yeah, the problem becomes debugging from the operator's perspective. If something goes on, for example, with networking, do you start at the, which container level do you start at? It just adds layers at which you need to figure out where you're even going to start debugging. So yeah, you're right. Ideally, developers wouldn't be able to tell. The gardener project, right? Yeah. You provide the Kubernetes cluster as a service. Yeah. The gardener itself is talking about 100% teach Kubernetes. So if let's forget about CF for a second, let's say, oh, actually, let's talk about the CF for a second. It has our space to take care of the multi tendency, right? But in Kubernetes, what's a common way to do it is maybe one team has its own Kubernetes cluster. Because once you get access to the cluster, you get access to everything. So a typical way is one team or one project, one cluster. So from that way, you can see, actually, it's very useful to provide Kubernetes cluster as a service. Okay, team may want one. Okay, garden, create one. There you go. Right? That's one perspective. Then the other benefit of that model I can see is take care of your control playing for your Kubernetes cluster. So basically, another Kubernetes is managing your control playing. Make sure it's running. It can be self-recovered, things like that. Another thing is, then your control playing and your working nodes are separated. So that way, enables, you give that team full access to that cluster without worrying, like mess up with control playing and all that. Yeah. Does that answer your question? Yeah. Thank you very much to make this real QA, real QA. I think we are running on top a little bit of time here, but please stop by Stuckman booth. I believe it's number four. So if you have more questions, feel free to stop by. Thank you very much. Thank you.