 Hello everyone. Good morning or good afternoon. We'll just wait for a few minutes to allow a few more people to join. Good morning, Quinton. Good morning. Sorry, I'm later. I had some trouble connecting. It's all right. We'll just wait a couple more minutes. Maybe a few more people will join. Good to me. Hello. Hi guys. Hello, hello. It's Philip. We're just going to wait one more minute and then we can start. All right, let's get going. We can catch up people from the recording later on. So before we start on the first item on the agenda, I wanted to share an idea that Louise and I had. Well, actually it was mostly Louise's idea, but I wanted to share an idea that or a proposal even about some of the format of the sick calls going forward. So we wanted to change around some of the topics. Today we have two meetings a month and often we've ended up having just a single meeting a month because project activity or project reviews have been slower than what's needed for two meetings. So what we were thinking was one way of revitalizing the sick and making it more productive for the community is to have a session a month. So maybe the first meeting every month could be focused on an information session. We would solicit the community to select topics or to propose topics and maybe we could invite a number of people from the community to actually present those topics and those topics could be anything related to cloud native storage, but obviously vendor independent as much as possible. And the idea would be to use it as a way of creating content which can be useful for the community in terms of the recordings that we create or presentations, etc. Luis did I capture the idea correctly or do you have anything else to add? Yeah just a forum where it allows a simple very low ramp of somebody just come in and talk about technology about storage and it provides a soapbox forum for that. That way we can share that technology across a large audience. What do you think about that, Quinton? Yeah some of the other SIGs, SIG runtime in particular is one that I've been involved in and has done this very successfully with guest speakers essentially and some of them are not necessarily from the Kubernetes community or the CNCF community but they get guest speakers from outside, other open source foundations, other projects that are not necessarily interested in being part of the CNCF but just to come and talk about their work and that's been very successful. I think the challenge is finding those speakers and finding people who are prepared to do that and who have interesting things to say, otherwise you can end up with lots of empty slots because you don't have the right people or you can end up with lots of not very interesting content, neither of which are outcomes that I think we want, but I think the concept is very sound. Okay that sounds great so in that case I will I'll pick an email out to the wider mailing list to solicit maybe some options or some volunteers for the first few sessions and maybe we can organize one of the sessions for either January or the February meeting and of course if anybody on who's on the call has any ideas or knows somebody who might be interested in doing one of those presentations of course feel free to raise your hand now or later on as need be. So the other agenda item we had we had today was something that was proposed by Rafael who is on the call and he was interested in discussing aspects of disaster recovery and how they relate to cloud native environments and cloud native storage and potentially with a proposal to either create a working group to create some content around this or perhaps to perhaps we could also add sections on DR to existing white paper for example. So with that I'll hand it over to Rafael to set the stage. Yes thanks Alex. No I think you summarized it well my question to this forum is if you guys are interested in discussing DR and creating guidance for storage you know storage software and stateful in general also stateful middleware since I think there is a lot of you know common concerns between storage and stateful middleware so the idea would be to create the guidance on what we think is the right way to design disaster recovery strategies in a container native world so in a Kubernetes world probably more specifically and I'm proposing this because of what I see with my customers which you know I work for that I am a consultant I'm an architect but I work in consulting I work with anything between three and five customers at having given time so I don't have a big you know sample but my sample is constantly varying and I see always the same thing which is people are trying to design disaster recovery people are getting to the point where they don't onboard stateless applications only anymore and we are getting to the point where there is interest for stateful so there is interest for databases and message queue systems but they are designing and of course the disaster recovery conversation comes up immediately once you have state once you have a state to manage but but what people tend to do is to design disaster recovery strategy as you would design it for you know the traditional traditional data centers or traditional IT with essentially active passive you know deployments and human decision to decide okay there is a disaster we have to flip the traffic we have to flip the whoever is the master and this kind of thing whereas I think the technology now allows to do much more and allows to create autonomous systems which can detect disaster and treat disaster essentially like an HA event where you know the the event is autonomously managed traffic is is is moved where where there is an active you know there is a working site the stateful workloads reorganize themselves autonomously with zero intervention so I think with modern approaches to disaster recovery you can achieve near to zero downtime and near to zero data loss but you have to do it right and and a lot of people are not aware of the options there is not a lot of knowledge about the cap theorem which which is what may you know essentially drives all of these conversations and and there is not a lot of knowledge about the middleware that are designed around the cap theorem versus middleware products that are of a previous generation and you really cannot be used to build this kind of systems so people does not do not even make this they are not capable of making this distinction that it's very difficult for them to have this conversation and I you know I have as you can see why there when I went to school when I went to the university the cap theorem was not had not been discovered we didn't learn these things there so unless you unless you actually proactively try to study these things it's difficult to to understand the big change that has happened in the meantime and the and the new options so I think if if all of this information was to come from a community like this you know a committee like like this like the six storage it would have more weight and it would help you know the wider community of practitioners or an open shift and well Kubernetes and open shift I'm interested in an open shift but in general Kubernetes to to do it right to do it right make sure that you're done yeah I was just gonna add one of this I was gonna add that what what in my experience are the difficult thing about adopting I'm finished with the objective but I obviously I'm pushing this in my in my day by day work and I can already share what is the what is hard about adopting this there are two things that are hard besides having the conversation in the right way with people you know you have to do a lot of education but there are two things that are hard one is you cannot build these architectures with two data centers you need three data centers and most of the customers have only two data centers you need three because of the leader election you know quorum issue but so so there needs to be guidance on how to build a third data center possibly in the cloud so so that's probably one of the highest expression of hybrid cloud that we always hear about right maybe you have two data centers on prem and then the third is on the cloud and that's how you you do zero downtime dr the other conversation that is hard is there is a lot of deploying you know existing deployments on old middleware that just doesn't cope with this kind of new architecture well and the migration is very it's going to be very hard so I don't know initially I think we're gonna have very little wins on your own very critical you know pieces of you know applications but I think eventually that is the way to go okay I'll stop here yeah no problem I this is my opinion so it sounds to me like you're asking to describe almost like a a solution right or a set of steps to create dr and how to set it up and how to do things like that I it's again my opinion in the cncf uh six storage I think if you took dr and describe what it is and describe what it entails and describes the benefits of dr as a whole right that I feel fits the cncf uh the storage landscape document better the actual instruction of how to set up dr and how to do things that sounds more to me like a kubernetes six storage thing or a or something that is part of the kubernetes group right uh because again this is just my opinion but the cncf is it's separated from kubernetes right it's more about concepts and instruction and more about so if you want to bring one of the things I've been thinking about is creating a multicluster in uh definition or what it means to have multicluster right uh now that we have a lot of multicluster kubernetes uh systems out there like anthos, tanzu, the one from amazon, arc and so on they require dr as for example so if there was a multicluster paper that describes what the cluster means and what it means to have dr in that concept then that's definitely I feel uh something that we could use uh but the actual steps to do it I feel that actual steps uh or like use this tool to do this and or use this project to do that that is more of a maybe a kubernetes sig project okay thank you for this maybe I didn't use the i words we can we can actually remove kubernetes completely from the conversation because this is really an intrinsic property of stateful workloads it doesn't have it does all the consideration apply whether you deploy it in kubernetes or or in vms but all the configuration all the considerations apply with regard to the what I said the quorum the data centers all of that perfect so yeah we can keep it absolutely general if that is the preference and I'm totally fine with that um yeah this is my opinion what do you guys think I think it's a fantastic topic I think it's arguably the general topic because this is not actually specifically about disaster recovery it's it's more about handling handling failures in storage scenarios and disaster recovery is kind of one one flavor of failure I guess I think this absolutely gets to the heart of cloud native storage I mean there are other aspects as well like scalability but but I think this is a great topic and I and I think it's as you pointed out a very misunderstood topic and I think it'd be a fantastic topic of discussion were you proposing to do that today or were you proposing to have a separate discussion or presentation or working group or what was your thinking that could be the first presentation no yeah it could yeah I mean first of all today I was probing this team to see if there is interest if there is interest I think we should decide what are we going to do about that right as as Alex mentioned in our you know private chat we we discuss a little bit about this I I envision a paper it could be an extension of the current paper or it could be a separate paper but something like that because I don't think this lands itself to a video or something short this this it's a complex it's a complex topic and we need I think we need to have the right medium is to start with a document in my opinion then we can simplify and we can extract you know maybe shorter and immediate messages but to begin with we should I suggest a paper so maybe then if if we agree you know if we agree then maybe the next step is to create a a working group to start drafting this paper I have I have already written a lot in this space uh so there are some things that we can probably reuse but my my articles are about doing these things in op shift so we can we can clear up all of that stuff and maybe reuse something on that that I have already written or we can start with original content it's for me it's fine either way I think using what you have as a starting point would be great maybe maybe reviewing that like one yeah either either do a sort of a presentation to the group to just give them an overview of what you've already done and then use that as context for starting the paper would be one approach yeah I'm very supportive of this as you can probably hear absolutely I look forward to it yeah I just just um from a from a logistics point of view what what we've found works is if we can kind of split the content into intersections or or or areas or or whatever that that we want to focus on um and then you know that that kind of lends itself to different people being able to contribute in on different parts of the of the document for example so so maybe maybe the first step would be if if you could put together even if it's even if it's just bullet points in in a google doc or something like that to to kind of say look I think the doc should cover these areas we can then have a discussion on the on the next call and go through um sort of scope out what those what what each of those areas could contain um and and sort of allocate and and see if you know certain people want to want to take ownership of of of some of those areas um and then that those those people can kind of split off and and form a small working group to to to get going on that I mean I for one would would would love to help with with some of that because it's it's um it's a it's a topic that's quite close to my heart as well and I you know and I kind of agree with you it's it's something that we come across more and more often as stateful workloads get split over multiple clusters and you know the concept of of of of HA now moves to to additional clusters I think one of the one of the challenges one of the challenges that that will that will have to kind of um understand is how much of this will be um a storage topic and how much of this will be um you know talking about multi cluster and federation and and perhaps other things like that which which um you know we we might want to try and sandbox off some of those some of those other items and maybe even invite some some of the other six to to contribute to those areas I was just gonna say uh quick thing um I feel that um maybe uh six storage is a probably a bad name for us maybe we should be called uh SIG appliance persistence so application persistence group um but uh I feel that there's two ways we could I'm now kind of deep diving into it so maybe we can put this off but uh we could do DR as area of failure domain like Quinton mentioned or we could create a whole new document that deals with multi cluster because uh and and the persistence of data reliability across multi cluster and DR being a piece of that right uh because uh again I just feel that we are in this point in the market where there's a lot of a methods of how to do and expand your specifically Kubernetes but your clusters across regions right and very easily and how that deals with data movement if one of the clusters die and you have to move to another which is DR right uh and what that means to application data right uh and I think that's something if we attack multi cluster and bring DR as part of it I think it's something that it could be really big because that's multi clusters first what I think anyway it's just my opinion but I think it's it's something really really nice I like what you said before Luis that that we should focus on storage and not assume that we are even running in a cluster because these are really is a is a topic of is an intrinsic property of stateful whether it's stateful storage stateful middleware and so we can say instead of talking about multi cluster we can say we can talk about the failure domain and you know so that could map to a data center or could map to a cluster it doesn't matter at that point we are much more abstract around the underlying implementation what matters is the behavior of this of the stateful middleware relative to the failure domains or what the failure domain does yeah and uh so I would focus on that and then I would and then we can have some pointers to work that other maybe other six are doing on multi cluster and say for for this to work in a Kubernetes multi cluster environment this is these are the capabilities that we need right and in my documents I have a good I think identification of what these capabilities are but we don't we should not try to even define out how those capabilities are implemented because there is an explosion of options right now and we just we would get lost so I think we need to we need to protect we need to do a little bit of scope management and stay I think in the stateful middleware storage discussion I totally agree I would I would caution against us creating the wrong impression that we're somehow doing multi cluster stuff because there is already a bunch of multi cluster efforts and they're much broader than this so so what I do think we will need to do is is definitely speak to the multi cluster people I know there's a sig in uh uh sig in the Kubernetes world that I used to run for a long time long ago um and I wouldn't be surprised if there's something similar in the cncf uh sig yeah there there there may be a group within an existing cncf sig that we should be talking to just to make sure that we know what prior out there is we don't want to go writing a paper that somebody else has already written for example um so that could be one one recommendation the other one is this term disaster recovery I'm a bit of a stickler for names because sometimes they kind of stick and then they get uh they take on a life of their own I'm not sure that what you're proposing is actually what people understand by disaster recovery I think disaster recovery and correct me if I'm wrong you probably know a lot more than this about this than I do but but disaster recovery tends to be associated with the old school way of doing this stuff where you you have a vendor who's like a disaster recovery vendor and they have you know warm standby data centers and all of these kinds of things that may just be because I'm too old and maybe that word doesn't mean the same thing anymore but I constantly I mean as you alluded to come across this people think disaster recovery means standby and means you know two data centers and all this kind of stuff and I think that's if I understood you correctly that's kind of precisely what we're not encouraging people to do and so maybe we want to come up with a better name at the beginning than you know storage disaster recovery and have it as you know durability or failure handling and storage or something like that I have my definition of disaster recovery but I don't know if it's a shared definition or not for me it's what you do when you lose a data center right and so what you what you said those are disaster recovery strategies and those are the traditional ones and they are the ones those are the kind of thinking that we need to change but yeah you can still lose a data center today so that event can still happen but if we want to call it some something else I'm fine yeah totally fine I I'm hearing more and more this term now that is modern availability which is somebody calls it like that so maybe that's that's what we could I'm okay with that what what we need to define is is to clearly explain the difference between HA and what I call disaster recovery but the main the difference and the way I invite my customer to think about this problem