 Hello. So let's give people a minute to join here or so due to the issues to meet in password. And I also put it into the documentation. There it is now. The image is put in there. All right, as always, I'd ask you to add yourself to the meeting notes. Seems to be. Let's give people maybe couple more minutes to join before we jump into the topics right now. Hello. Okay, then let's get started. Let's see how people are showing up. So for today, couple of updates. The first one is on the operator working group. When I joined the last couple of meetings. We decided to merge the working groups back into the main meeting here from the presentation working perspective also because they are at least some of those meetings that did not really have didn't have a lot of attendance we also had hand and all together so we thought let's move it into the main meeting. Also some of those meetings, I tried to join them nobody really showed up to those meetings will be tried to put the work back into one meeting. And we can split things further out as we know, but obviously people can focus on what they interested in and this is true for both the air gap work, as well as the work on the the operator definition. The first topic. I want to work on. I look into is that that working document we had on the the operator definition. So this document has been around for for quite a while now with. I'd say limited progress on the document per se. I last meeting I took the action item to comment pretty much on all the action items that were in there in this document. And to really move this forward so this was also requested by that you see that we come up with a better documentation for a better definition of what an operator is. And I just briefly take the time to go over a couple of items here. So the first discussion item was that this related to technology section in here. It created some kind of confusion that people said well this is not necessarily presented right not all technologies are in there. And the last time you already discussed that for the related technology part it's not even really necessary for this document so the goal of the document to describe what an operator is, but not necessarily to define what other possibilities are out there to run different technologies and what could be used as an alternative to some of the capabilities of an operator. So the proposal was to remove the section this has been now around for almost a month to remove the related to our technology approach session. So the question is that anybody object which has removed the section really focus on the core definition of the operator. I think it would make sense to describe some other technologies in short sentences but only to distinct between the operator and other technologies. So I think we should we should define when when an operator totally makes sense and when does it make no sense when it's when it's better to use charts or other things. But is it really our job to tell people when they should be using which technologies when that main goal was just to describe what an operator is. No I don't think so. And that was the driver because I'm that this was like Matt's comment here. On the help side this is not an accurate description of what help is doing which I might agree with to some point, but nobody. So people were complaining okay this is not accurate enough that we will put it in there. At the same time there was no update so my point was let's focus on describing what the operator is how it works or how it's supposed to work. And leave the section out because it created a lot of confusion in the past. This would make total sense. And we have some others in there and if somebody wants to take the section to a level where it's fine we can leave it in there so Thomas if you want to take it. I could try it until the next meeting. Yeah so please get in touch with Matt whether he's fine and maybe some of the other project I think we don't necessarily need it honestly. I think it doesn't really help us. Just taking it out here. So I'm not sure if someone read it. I've posted something on the mailing list to give to document a bit of another structure, but didn't get any response until now. So, yes, I think I'll also discuss this with Matt. Okay, so you get the action to remove it or not. Is there what do the other people think should we have it in there should we remove it. Maybe we can also discuss your offer later on in here to provide a link to your document in here as well. Okay, so let's look at some of the comments in here. This I want to later on discuss a bit. I was struggling with this. So this was like a feature creep. Most of this document. If you want to put it that way that say who's the audience of this document and we started while the TUC requested it which would mean it's a very technical audience obviously we extended it to to technically kubernetes app developers and cluster operators. Which I think also makes sense if you're building an app, or if you're operating an app on a committee discuss the you should understand what an operator is that they totally make sense. What I kind of is totally out of scope for me a director of a non kubernetes company to understand what an operator is. I was back then part of this meeting but I think we're still struggling with the technical definition of what the operator is. So I remember the back to the day when we had this discussion was yes my boss he's not like super technical I want to implement an operator and I want to give him a document that he understands why I want to do this. My proposal is to remove non kubernetes users from that operator definition, especially if you don't understand how kubernetes works it's like very hard to understand what an operator is. And that I agree with that that that is very weak because director in a company could mean different things depending on the company. It is to remove this as a target audience, I mean if we could if we want to have an operability guide later on. I'm fine doing this but it just makes like super super wide. So anybody okay so then is there any reason why we are explicitly calling out at developers only in the line number two. Why not, you know, DevOps sari kind of a person us. And as the users to target audience. Yep. I mean I was at the CNC FTC is is a target audience but they may we're writing it because we're building it for that you see request. I think that makes. And so, and I removed the technical here I think it's kind of like repeating itself. I have for, I have luckily really seen on very few untechnical app developers cluster operators and you said SREs. Yes. I'm also if you're fine I'm moving this into FTC to explicitly mention it. I think it's not adding any massive value here. I mean obviously it is for the TSE but yeah so just that that I ask for your permission all the time if you do not agree simply speak up or agree if you have an opinion, then I think that that would be good so this is you so you're either developing an application you're running a Kubernetes cluster or you're an SRE that is obviously responsible for an application that is running in Kubernetes. Good. I know this sounds like baby steps but it is some progress here so and then note in here. So, so an operators in the in cluster application which uses declarative paradigm for the installation configuration and operation of an application that was a comment that that Tom put in there and the working group so I think the only discussion in here is what about operators that are built for non Kubernetes cluster resources, something like a CK. I don't know if you're familiar with a CK but a CK is the operator that allows you to manage Amazon easy to or AWS resources. I think this is okay. Well technically they're not calling it an operator they're calling it a controller for Kubernetes. And I think that the bigger question here is, I think how you look at this conversation with Cornelia last week in the operator meeting because I think that both of us showed up in that meeting and had some discussion. I think it's how you see Kubernetes do you see Kubernetes as more or less as a platform to build platforms and operators as a core concept in there. Or do you see it really as a means to run container type workloads. Okay is an example here is maybe this is something we could put Thomas in the related technologies. They even say that it's just a control they don't name it to call it out as an operator because it's not managing anything inside. I think this is similar to cross-plane isn't it. So also just a set of controllers and control plans to build other other platforms or to control other other things. I think it's, but what I still like to do in here that sentence per se is not wrong. Just want to operate this help its consumer to manage the life cycle of the resource without the need to run any code. And we can. So, because none of these comments had like a lot. I would really move the out of cluster discussion to I would rather make a separate comment for this one. Then I was it goes deeper into this. Yeah. Can an operator be used to update in another operator. Theoretically as I think so. Should we include that then because it's only talking about applications. Unless we start call operators another application. And if you're. I think if we could be more precise what we mean by by an application for instead of kubernetes because in kubernetes there is nothing that's an application. You have a number of resources that you can update in kubernetes and the application is none of those resources so technically you would define an application. You would need to have a custom CD. Yeah, I think that that's actually a good point should be named what what you can actually control what you should be controlling with it. I'm just adding this here as a. Sorry. As a comment. I think you cannot double comment. I think another application could be replaced against a set of objects or objects or however you want to, you want to name it. I think we should name the objects that you usually control by like. But that's a good point. We should really call it out because an application per se does not exist. So, and I think then it goes into okay what an operator is not meant to do monitor an application they can observe and utilize metrics lots of events to support the proper function. But they're not used to do that in a deeper sense. So yeah, I already commented on this one I'm not really understanding what we mean obviously there is a reconciled loop, but it's not a log shape or a log agent per se. I think this relates to the. But technically I could explore. I could also write a CD based on the resource. Yeah, it's not replacing a monitoring system. What is this coming from is this operator capability model that was initially core as than redhead resistors like phase one, which is basic installed and there are seamless upgrades for life cycle deep insights and and auto pilot for for an operator. Now, now that I read this document a bit. With some distance, I feel at that point in time I still have no idea how to what to do how to build an operator in kubernetes. I think it's very abstract but I don't know what it is how to build it like for my if I'm the developer. I still don't know what I should be building right now. I mean we know you want to have a custom controller that is implemented. You should have a CRD for this customer for the custom controller. There's a recon sound loop in which you should react to what you're doing, but none of this is anywhere here. So I don't know how you feel about this but this is, I think we might want to go like one level deeper like this is if we talk about kubernetes operators context of kubernetes. This is the core part of the kubernetes architecture that's relevant to operators and this is how you extend kubernetes with like an operator or how you add additional functionality in there. And obviously there's there's multiple toolkits out there which you can use to build an operator. But if you look at the initial definition in there is because there is one in the kubernetes documentation, operate the pattern here it is. There's the aims to capture the aim of a human operator who is managing a service a set of services. Human operators look after specific applications and services have deep knowledge of how the system ought to behave how to deploy it and how to react. If there are problems people who run workflows on kubernetes often like to use automation. Okay, that's fine. Operators in kubernetes kubernetes is designed for automation out of the box you get lots of building automation. You can use kubernetes to automate deploying and running workloads and you can automate how kubernetes does that kubernetes controller concept. Let's you extend the cluster behavior I think this is something we should put in there you extending what the cluster can do. So this is I think number one that's not in there it wouldn't be there. The operator that doesn't manage this is a way how you can extend a kubernetes cluster with additional capabilities. You can use that's the extended cluster modify the code of kubernetes it's without modifying the code of kubernetes itself. Operators are clients of the kubernetes API's that act as a controller for a custom resource. So okay I think that's I think that is a bit more specific but what I think might be helpful is like having a picture like usually when you have an operator like these are your major building blocks that that you have. Like your custom resource your controller component these are like this is your reconciliation loop or this is your interaction with the kubernetes API. This is where you basically build almost like that there might still be like a 10,000 feet view of this but a bit more focused on the developer here. There's a couple of more examples down here like deploying an application on demand taking a restoring a backup. Handling upgrades of the application code along several changes without publishing a service with patients. Simulating failure a custom resource that's another example is a custom resource and sample to be that you can configure into the cluster. A deployment that makes sure a part is running that contains the controller part of the operator. A container image of the control code that queries the control plane to find out what the sample to be. Actually this putting this maybe in a graphics might be in a graphic might be a nice way of actually describing it. So first you have your custom resource. Then you have a deployment for a part and a container that actually contains Europe. That contains your controller. Here we have the container image. Controller code that queries the control plane to find out what sample to be resources are configured with consideration loop. The core of the operator code is to tell the API server how to make reality match the configured resources. If you add a new sample to be the operator sets up a persistent volume claims to provide durable data storage a stateful set to run sample to be and a job to handle the initial configuration. If you delete it. The operator takes a snapshot then make sure that the stateful set and volumes are removed. The operator also manages regular database updates for each sample to be resourced the operator determines where to create the parts that can connect to the database and take back up these parts would rely on a config map. If the most common operator is to add the custom resource definition it's a little bit controller to the cluster. The controller will normally run outside of the control plane. If you would run any containerized application for example you can run the controller in the cluster as a deployment. So I think that sample to be this list by the way they also extended to think that older documentation didn't have all of them in there. This cooler cube with the controller. And the operator framework in there. But I think putting something like this in moving something like this over into the dock. That would actually be helpful. I think now I understand by like even having a picture that okay this are the components that you have this is what it's doing. So, if you do not disagree I would move this part over. I think we need to then see how it better fits fits in there. So this is the whole discussion of people don't like phases, but again there was. For example last year, no more comments. So I'm just closing this one. So what I, what I think would be a good idea is we have like this high level description if you could have if you want to use the phases. We have a description how what an operator usually always does in the individual phases would like as per our sample to be example do certain things. Like how would it handle an upgrade. That's the only way to do it. But it gets more interesting when we talk about for life cycle, especially also the deep insights bits and pieces how it would do that. And also like the autopilot type of a project, having one example what it, what it could be doing. Does this make sense. So, would it make sense to define a sample use case at the beginning such as the sample to be. And then, and then describe the describe the whole operator thing based on that. Yeah. That's why I hope it over the sample to be the question is do we just take sample to be or do we need a better use case. So starting with sample to be is not the worst of ideas as we're going to need to start it with sample to be as well. I'm putting it in the notes. This could be an example application so we just need to this have one in there we could also have an application that's more complex. Maybe one that has a state folks that has some additional resources it's managing. Yes. Okay. Do you want to write it Thomas. This is also almost a year old and April. So the main takeaway was the way it, the only way an operator right now can, what is monitoring application is why are that the Kubernetes API. So everything you can query via the API it would know about that if a pod died or if something happens. But I think an operator can monitor a bit more because you also can talk to the application itself. So you could monitor have that points. Yeah. Okay, good morning. Should we have a separate point maybe on the monitoring piece how and that an operator could monitor it yet. Some kind of absurdly the features are into the operators the case far as I know, I think it would make sense. It's greatly confusing for us to say that operators are not meant to then monitor an application then talking about few things to monitor. I mean I feel like it's, it's confusing saying that it's, we're kind of conflicting ourselves in that statement. Yes. Yeah. You mean because further down here we talked about that we have posted on monitoring the application. Yeah, especially if it is one of the key faces. Yeah, I think the idea was that they are not replacing a monitoring system or like an observability system that you're running. And they only monitor the application to the extent that they need to do their job that they are designed to, which technically doesn't them limit them doing anything really. I'll put this in here. It's confusing given. Yeah, maybe we'll remove it. Potentially we can add a, I don't know, something like the sample operator that we have we could add a use cases, and in that use case section we can see that not meant to use in this like prolonged monitoring of an application in communities. But aren't the use cases the faces, more or less like how do I do, say like full life cycle management and if you zoom in a little bit, you would have like backup failure recovery. What do you think of the other. Yeah. Yeah, but it's a good point to describe that for this use case how you would actually address it. So I like the idea what they took, the way they took it in the company to stop examples on making examples just capital case that this is not the only way it can be done. It's just like really one example on how things could be. Yeah, this is just the same model in a different way. There's just two ways on how to present that model. Okay, here we have the implementation that we discussed before like the technical implementation. Just a quick question on the two diagrams should be include both the diagrams because the second one to me looks like sort of representation of the current population, like basic install is high and usually we use we use diagram like that to show the population. Not the life cycle or faces. I personally don't think that we need both and get just repeating themselves as well so to me we're removing this one. Yeah. So I'm just leaving it in there in case anybody wants to have it in there and I'm not expecting it but just let people comment on it. So what do you think about this one so I know we had this discussion a long time back we said what we have to distinguish operators from controllers. What do you think about it if you really go after use cases this is what you can use an operator for this is like an example of able to use it to do different things. I don't really understand why, why we should differentiate between operators and controllers, because technically an operator uses a controller and it's not another is not competing with a controller. Here's a question then, why do we even have the term operators, if they're going to be synonymous with controllers. What's the point. There has to be a differentiating factor for it to be a useful meaningful term. What it loses its original chorus purpose. If we don't distinguish it from a controller where we're, we're making it so different from the original purpose that it'll probably end up confusing a lot of people as well, because it's gone so far away. There's a controller that uses a shared informer against the core object. And could you argue that that is an operator. Just a, or, you know, just a built an object. Is that an operator then. There's a difference and I think it needs to be outlined. So basically an operator is not the controller that's true, but it's, I think an operator is just an enhancement of a controller, because it has additional features. So, so what is the difference and why I mean people need to know that. Right we're trying to explain it to people in plain English and they're going to see controller stuff all over the place. You know, distinguish between the two they're going to go what's the difference. It's why this question even came up, and we say well there really isn't any difference they're going to say, Why do we have the term operate and then other people are going to say, but chorus define this in a way that's very different than just a controller. Why are you redefining it to something that's just so general that it can be a controller. It's going to raise a lot of questions if we don't answer this. I get your point so I think we should describe a bit about this. But the question was what's the purpose I mean the way if you again look interestingly at the Kubernetes documentation. They talk about the operator pattern. So it's as small as a pattern how you can manage workloads they don't. They provide insights and how you can implement it with Kubernetes. Platform mechanisms but it is a pattern on how to do things rather than it is like an actual like core artifact of what what what we're doing and eventually people are releasing operators. So if we look at here like where the original documentation comes from it just specifies the operator back pattern. So, so Matt do you have a proposal on how to phrase this so that it helps people to say I mean, operators are a subset of controllers that do a specific pattern. And that pattern is outlined in the definition of what an operator is. Yeah. There are some controllers that implement that pattern, and there are other controllers that don't implement that pattern. And I would probably give some examples of controllers that don't implement the operator pattern to distinguish them. And an operator pattern by the way isn't just controllers right, because an operator has to extend the API and a controller doesn't extend the API. So operators a pattern also needs you to extend the API right. Yeah, whoever has good ones obviously comment on like good examples but I'm putting this in here. Yeah, then we are again at the related technologies and approaches section which Thomas has taken the action item to talk with the individuals whatever we want to keep it I think we don't necessarily need it this is not the competition here. Just help us to understand and describe the concept and help people to use that concept. And not say this concept is better than other concepts that are out there. Good. I think there's some action items in this one for the next meeting to get to give it some updates on some overhaul. I know this is kind of like a bit painful to get started with think that this will then get be more say rewarding once people who were looking at learn more about it will reference this document and say thank you to us for eventually having it put together. But I know it's. It's a bit of a, like say the challenging has to get it done in the beginning. Okay, we are almost out of time especially since we had to be able to come here a bit later. Just a quick update the app delivery landscape that Harry was kind of initiating as like not long and other will update as to I would move this to to the next meeting to talk about this also on the air gap and the air gap working group that Calculus on I reached out to them once I'll do it again for examples on the air gap environments and the work that they'd like to see. The last one to take four minutes to talk about this one so Harry and I for this also came out of the last meeting where we discussed well okay having a landscape is one thing but you know many people today are explaining about since they have landscape and it's and you know there's even parcels of the CNC and landscape that you can buy because there's so many things in there. Okay, let's build like a simple sample application that people can get started with somewhere they can use different technologies to try out okay how like a delivery use cases with it and so Harry and I decided okay let's also submit this to KubeCon and like show common examples and show it was a demo application. And last time I took the task to look into the demo application the obvious one would be hipster shop because that's the most widely used my takeaway so far however was that a hipster shop is nice to show a microservice application it means it misses a lot of things that are kind of crucial to to more real world type of application like there's no stateful workloads in there there's an issue for this. There's not really a database in there that needs to be upgraded and managed maybe even a database that itself is managed by an operator like I don't know like every test or something. Obviously it's all like one version in one here so there is no blue green type of deployment there's no cannery releasing capabilities in there by default. There's also no third party dependencies that might be requiring any secret management. And there's also not multiple stages with multiple configurations so it's kind of unsure how you would handle like this application running let's say in a pre production and the production scenario with different secrets. I would define it there. Okay, can I just also to throw another layer they've been the end users have been putting out their radars lately. And there are certain things that they've put into certain categories within those radars or whether you know use it trial those kinds of things that have been typical to radars. I haven't looked but does the hipster shop example actually follow the same patterns that they've talked about over in the radars, or is there a mismatch there. You mean on the technologies that they're using in hipster shop. Yeah, and in the way they deploy it and what they use. Does it actually follow what the end users have said, these are the things that are in the, why the adopt phase. Does it actually follow that, or did just doesn't use things that are maybe more niches that the developers of hipster shop wanted to show off. Don't think that they want I haven't looked at it but it just occurred to me. Yeah. So, interestingly, this is just the continuous delivery landscape data one would be the. So what you have in here in the adopt phase, you have flux and help in there. I'm not sure what it uses held by default or reduces just manifests think it uses both. And interestingly, if you look at the rest in there I mean there's Argo in there in the SS phase obviously it's not using anything Argo related. Jenkins X and spinner person not in there as far as I know you could probably use it. You know it's tecton. Don't think so. Well Travis you could use traffic with it. And flex in general also points out get ups is the deployment model that they talk about a get ups deployment model in hipster shop. I check it out. I think I will have to check whether you can deploy in the get up a bit of session, because you know, if this is just one of those things I mean if the end users have said this. Then maybe it's worth looking into what kind of example shows off the patterns the end users already have. So we look like we're on the same page and we're actually not just showing off cool technologies in the sense that we're actually showing off that the end users are saying are good ways to do things. And so I think it's a nice way to have synergy because then you we can actually say in the thing and the end users were talking about this and you'll see the same technologies, you can reference other things happening in the CNC often show how they jive with each other. And so it's an opportunity to reference. Yeah, I think that's also what we eventually want to do so I think we could also file request tickets or something if there's a better application out there. I'm more than happy to honestly look into it but I think putting something together for a presentation I would do two example apps. I would do one that is a microservice app, and then I would do one that's a lift and shift where you're taking a legacy app and running it inside of Kubernetes, because you end up with both situations a lot in what people want to do and how they've got to deal with, because a lot of times there's a lot of people just bring over their big thing, run it in Kubernetes and then break parts off of it or just keep running the monolith there. So sometimes having two things that show you here's how these technologies and ideas can be applied to two separate things. You don't have to break it apart in the microservices right away gives people real world path forwards that they're trying to work with. I don't have an example to tell you to use right now. I'm just taking them. I just think that we should start maybe with the cloud native one first to live my life. But as we as we have more, more more end users that also will require is okay how can I move something, but we just gets a bit more complicated because then we also this 12 factor at discussion maybe was a feed around that certain things don't work. Worked that well. I think the way it helps approach is interesting. Also the state for workloads. I think we have to have to mean that is if it's totally unrealistic to run an application that's not running a database. Well, sure, I mean, you can always take something like the Reddit approach, which is where they ran their databases and VMs and they ran them separately and then they ran all their stateless services and Kubernetes, because they weren't ready to do that migration so they did a partial I don't know where it is today if they've migrated their their stateful workloads but that was one way, and it helped offset folks as they went through this with their different trust. They were very happy to do their state for their stateless stuff in Kubernetes and then as they built up trust that you know you can move over your stateful stuff. And I've seen a lot of people have those trust factors and so I think it's good to have stateful stuff, but also think it's good to talk about people as they build trust in these kinds of shifts to be know that there are options for them in the world of shifting. I'm putting it in here and interestingly for stateful workloads there is there is an open issue in in hipster shop. I think I found taking it as that reference application and we also use it internally for some demos if you really run it in multi-stage environment is actually consuming quite a lot of resources. Like running it in a three stage environment requires quite some cluster resources obviously. So maybe something a bit smaller might be easier for people to to experiment and use CNCF technologies. But I totally like the GitOps approach actually would be nice one to see what it can actually be deployed with flux. Or how far it is a way to be able to be deployed in the GitOps mode. It should be easy. Okay, I think we are already beyond on top of the hour for this one. So yeah for the next one there is obviously some work on the operator working document. Some of that the landscape work and on the other landscape. I'll work with our team over here to see whether we can actually use it like in a flux type of deployments and use other tools in there. I mean Helen should be an obvious one. You should be able to deploy with Helm if you can deploy with manifests. But also what else they have in there from from CNCF technologies. Alright, then thanks everyone for this meeting and see you again in two weeks or in the chat and on the mailing list. Bye everyone.