 Hey dog, Shrikant told me that he wouldn't be able to make it, but it's fine, we shot it. Yeah, yeah, yeah, yeah, so we shot it and he's fine, for example, with what I had been sending you through email. Hello, hello. Hi. Unfortunately, we need to keep a few more minutes. Hi Matt. Hello, hello. So, let's get started, I guess. So as per usual, this call is already recording, but everyone is free to join without me without talking or even under a different name. We don't judge just fi so you're aware that that all of this is being recorded. So, let's start with Santa's due diligence. I hope everyone read this document. Who read it? Make sure of hands, I see all of you. Two or three people. I've been off the grid on purpose without even a laptop for a couple of three weeks almost, and I'm just back this week. So I have not read it. I think we have a better ratio than last time. I think we also have to attend these left so that improves the ratio even more. But I think we actually have a somewhat decent rate of performance. I just need to get something from my throat. So I also throw it into the chat so everyone can just jump into the document. As with the last document I would highly suggest we try and walk through at a certain pace. So to make it explicit, everyone. Everyone should feel more than welcome to raise objections or anything at any time, no matter if this is your first call or your 10th call on this call. You should feel comfortable actually putting your, your comment or your name or your agreement on this. That being said, I'll still try and make it quick. So, yeah. Should we, I can screen share if you want, that actually makes sense. Okay, I hope you can see my screen where I'm locked in with your user to make any edits. Yeah, can see. Let's walk through this as per last time. We know that Thanos is self governing. We know that there is a provision in place to limit take over exempt by any single company we know that finals has a code of conduct. Should I open it or is everyone happy with just acknowledging that this is the case what I just talked about. Good. So someone put in the comment section for me. Thank you. So let's play call for consensus. Sorry for the commotion in the background. The little ones are having that much. So call for consensus sick observability is happy with the section above all agreed. Anyone not agreed. I think we missed the first one. Government. You put this in there. Okay, sorry. Okay. So for that same call for consensus, my suggestion is sick observability is happy with the section above. Are there any production deployments of Thanos. Again, I think most people on this call will be aware of several of the production on deployments but if you're not. I think probably is still using it. They're using it in production. And they have quite a large installment or installation I know this by myself. Alibaba is also not, not small right head neither. So long story short call for consensus same suggestion as before sick observability is happy with the section above. Anyone agree. Anyone disagree. The next section is about the principles and alignments with the principles of the CNCF transparency and guidance and technical governance quality. We already talked about the governance above or at least briefly and I read it and I agree that it's that it ticks all the boxes. It's a duplicate question which we need to suggest to you see to to change the template. Same here. That's also do and same here that's also do so we'll be walking through those in a second. The question if it's useful for cloud native deployments. I think we can agree with this, of course, it's obviously following the premises and that's the Kubernetes way of doing things. And if it has an affinity affinity for how CNCF wants to operate I would also agree with this as it's literally the previous way of operating just with object storage to allow for scaling. So same call for consensus I would once again suggest that we put in sick observability is happy with the section above. We don't want to read any disagreements. So, this is about the design of of Thanos and that all components will not burn down and explode if put on the load reliability concerns, or should we just like. So quick show of hands or do people want to read this. So we can just pause for like two or three minutes and everyone can read through this, or do you just want to walk through this at pace, both is fine by me I just want to to feel you. Even if you think your opinion doesn't matter your opinion matters your modern welcome to speak up, you're actually invited to speak up. Yeah, I can only can someone just say something cause I can't see video. Yeah, I can hear you. Like, I don't know I think like most people who who actually read this document sort of wasted their time right so I don't know. Maybe questions because people commented. I guess that's what I wanted to do anyway and guess it's a good point but I want, yeah, but then we just we just do what I intended to do course. Okay. So, I would be happy with our tax reply to to that question. Maybe maybe put put in exactly this reference that would be my personal feedback. Got on heads like very fair questions and the truth is, is like kind of the thing that that is in progress as well. But it's kind of moving things are moving towards improvement, but yeah, fair questions. The point is to like this, what I just wrote and marked. Yeah. Of course, then you can also resolve the question basically and you you document what what you replied for everyone else. So the next question. Once again about scaling out. And this is already done so. Is it merged. I actually didn't check. I mean Brian said look good to me so it's like merged already. Personally, I'm also happy with your client link to. So are there any other questions or comments regarding this section. So just for the record I read through that I was happy. Call for consensus sick observability is happy with the section above all agreed any disagreements. Next day section is about usefulness in cloud native diplomas and to the degree of of being architectured in a cloud native style which is one of the questions which will also see improvements suggestions from the sick, I would say. I read through it. I was happy with the replies. Anyone else with comments regarding this section. So once again call for consensus I would suggest as you probably expect the same wording sick observability is happy with the section above. All agreed any comments disagreements. Next one. There is an affinity to how seems if operates and that the expectations of being the scf project are being understood, which kind of gets a default. Yes, of course, there is quite some overlap with with the permit this team and as such. Basically, that team grew along with scf. So, personally, I'm happy with that section. Are there any comments or question regarding it. Very good. So, call for consensus once more sick observability is happy with the section above all agreed any disagreements. Next one is about being used in production, which is a do to the above. So, I suggest, without a lot of discussion, the same call for consensus, but you can discuss if you want sick observability is happy with the section above. Very good. Next one is about healthiness of cometers and maintainers. Looking at the numbers. Obviously redhead invests most, but there is a healthy, a healthy distribution of contributors. So, in my opinion, we can make the same call for consensus sick observability is happy with the section above. Any disagreements. Very good. So next one. Demonstrate a substantial flow of commits and merge contributions. I would maybe suggest to also put a screenshot in but I mean we can even mentally reuse the screenshot from above. But looking at the links before I would be happy with that. Any questions or comments about about the distribution of comments and mergers. So call for consensus sick observability is happy with the section above all agreed any disagreements. Nice. The CF alignment. Bartek remind me is this also a copy and paste from from the initial thing same as with cortex. That's correct. Okay. We lost right. Yep. Yeah. I expected as much when I read through it, but I didn't I didn't actually cross check. So, given that CNCF already accepted this section above, I would suggest the same call for consensus sick observability is happy with the section above. All agreed any disagreements. So there's a check if there is a design document and then overall design, which I know is available. Yeah, let's not look at it. Any, any questions or comments regarding the design doc of of fellows. So, same call for, I see someone clicking around if you need time just say so no problem at all. So call for consensus sick observability is happy with the section above all agreed any disagreements. Very good. Yeah, cloud native use cases. I mean, it's kind of obvious. Yeah, for, for basically the whole section. The fact that infrastructure management and such is outside of scope is in my opinion completely acceptable. I'm not putting any, any non primitive services curry mechanisms on top with in the context of Thanos is also fully acceptable, especially due to the overlap, which would mean that if Thanos needed anything more. The project itself would be more than capable of implementing anything missing with in premises upstream, and then basically use it back in Thanos so the scope is also completely fine. So, same call for consensus. Any questions regarding this section any objections, if not, and again, please speak up, of course I keep talking no one else is talking. Good. So same call for consensus sick observabilities happy with the section above all agreed any disagreements failure modes, and if they're being understood. I'm happy with, with everything which which I saw. But this is actually one of the most interesting sections. So, again, if you have any comments, any concerns, if you want to take some time to actually read through it, it's all fine. You're more than welcome to just just say so and. Okay. Then call for consensus as per usual sick observabilities happy with the section above. All agreed any disagreements. I'm happy with the level of detail personally. Yeah. Me too. It's I like how how it's structured I like how it's being thought about. I think it's probably out of scope for the specific question but I'm curious if there is either. Are these things that are tested for in terms of regression. But again, I don't want to torpedo anything. What's the correlation between these failure modes to what's something that would be caught by automation. I don't think it's like this is how we can learn and get some feedback like, yeah, I can essentially say that we try to test against those definitely and especially in terms of regressions, we immediately kind of create a test if we found some certain back that is happening and that we fix just to make sure we're not kind of repeating that pattern. And now for this, I think, for, especially for those distributed system fail overs, it, it is more. There is definitely area for improvement to get more testing and particularly we are thinking about the receive component which is replicated and, you know, everything has to be reliable that if one node is down the application will will cover you know and and and this this note will be joined again if it's up and for those things we recently think about, you know, kind of have some Jepsen style testing. Are you aware of Jepsen this is like a nice project for that fault injection in different areas. So we are definitely thinking to kind of expand on this and yeah I don't think it's it's well done in in any kind of project in CNCF right so definitely some room for improvement here but we try our best. I just add to that, like I remember we already wrote a bunch of like test which tests timeouts and things like that basically other weird failure conditions. So we have definitely have some amount of tests that were like, I don't think it's obviously 100%, but yeah, we try to do our best. Matt, is there is any is there anything more which you would like to see handled here like for example explicit statements about all this or something. Um, I don't think personally again know I love that this is called out in in the assessment. I think for a high level question I think the barrier as I understand out the barrier but the, you know, to go from from incubation on to the final phase is where I would personally expect that kind of thing to be more complete. I mean, it's great. I mean this is a deep level of analysis of what the failure modes are and it's something that when published developers that want to contribute to the project can use as a starting point. And a lot of this isn't at the end of stocks as well. I was just more curious. I guess a very good feedback. Yeah. This is from sandbox to incubation right so this is, this is not like, you know, the end state, ultimately, finally, but thank you for documenting it at this level of detail. Other comments regarding this. Okay, so call for consensus. Also then suggestion six observability is happy with the section above all agreed any disagreements. Next one trade offs regarding performance scalability complexity reliability security etc pp. And if those are explicit implicit if anything is tunable. I also read it. I'm happy with it. There's one comment which I have which is relevant to the section but not relevant to this document which is why I didn't put in the comment. And I want to talk about this after we're done for maybe 30 seconds but just off this document review, I would once again suggest sick observability is happy with the section above all agreed any disagreements. Any important holes. Hi. Like, sorry, I joined the call late, but I think we should also highlight on the security side. I think it's the question that at the, like, at the bottom, so we still didn't got it. Okay, because you made a comment we will I think we'll talk about it. Okay, no shoes. So you're happy to offer. Yes, yes. So call for consensus sick observability is happy with the section above all agreed any disagreements. Important holes. Yes, the hand charts. I mean this this question is kind of weird course, obviously, the people who wrote this documents are the only ones who are qualified to actually judge the answer. So I dare say it's not a very good question, but that being said, within the scope of what we can do within a document review, and I'm not implying that there are any holes which have been tested over I'm just saying I can't. I don't feel I am able to do diligence on this beyond trusting the people involved. Yet. Call for consensus sick observability is happy with the section above all agreed any disagreements. I'm sorry, I think that him wanted to discuss point five. So we already passed that I was wrong. That's fine. That's fine. Okay. Yeah. You mean in this section which is which is forward referencing. So five year almost here security right. You mean that this can also be part of the question for for the below. Yeah. Okay and here is no specific reference. Got him you want to ask something about this right. Yes. Like for multi denancy I like it's not baked into the project it's a additional thing I was just curious if it's possible for users to abuse that. I mean, abuse that it's a bit hard because we rely on proxies right so basically if you can abuse trick proxy into doing something. Then yes, like, like no if your proxy is behaving well which is outside of our scope. Then no basically proxy will ban you from like sending traffic to turn us right. So that would be like really hard, unless like, I don't know. It's, I guess some kind of you know we don't do like query parsing or some kind of like you know, basically like. It's just like a typical injection type of things which I think it's really hard right. And like I know we want to move into Prometheus community but do you also are looking into moving into Thanos itself because if like, I think you should claim multi denancy as a core competency of Thanos as well and having it part of the Thanos organization helps that in stuff it being in a different organization. Right now it's sort of, yeah, Prometheus community organization, whether it should be in Thanos. I guess we need to discuss it with the containers. Thanks for the suggestion. I'll put it on our agenda. Yeah, they're very good thoughts. Yeah, and I think we're definitely moving it somewhere, but I think the plan was Prometheus community. Would that be okay? I think it's still kind of controlled kind of by Prometheus system, I guess. I think the idea is that we should move it away from that and claim multi-tenancy to be a core competency of Thanos as well, that's just the feedback. Because this is the first time I'm seeing that Thanos does multi-tenancy while reading the doc. Yeah, but like, in fact, you can make Prometheus multi-tenant as well, thanks for this. And I agree, it's, well, technically less secure than in process authentication and like multi-tenancy. So you need to kind of secure your container, your runtime environment. So definitely that's some trade-off here that maybe we should point. But definitely the global as well. I want to mention also that CNCF offers us some security outage. So hopefully we can have another kind of very deep look on that. Yep. I'm happy with this, like, what's in the doc. I just wanted to point that out and discuss that a little bit more. Yeah, very good thing. And jumping back towards the context of the explicit document review by adding more stuff and making things even better and more explicit. I don't think we need to revisit previous sections and can just continue. But if anyone disagrees, please speak up. So next one, code quality. Well, it's Bartek, so maybe we should just not pass them. No. No, in all seriousness, with every time I actually did look at code, it was decent, so. Okay, man, what project wrote its own code style for goal line that has some official one? So I think that's kind of exceptional. Yeah, I know, I know. I've been not modest. I'm spoiled by the pro style guide. And I know Tom hates me for saying this, but it's a really nice style guide. But yeah, I know you have a style guide and I like it. So within the context of this document, I would suggest the same call for consensus that SICK observability is happy with the section above. All agreed, any disagreements? Very good. External dependencies and other dependencies if they are being justified. I actually, I thought I had put a comment in about, about stating that you depend on rendering in Prometheus stuff, which is totally fine. But I thought, I just thought it's missing in here. Of course, this is a significant part of the Thanos package, as it were. But beyond that, and you can obviously discuss what dependencies are. I thought I wrote it in probably it's open in some common box where I didn't pick something. Yet within the context of this, so maybe at this, Bartek, can you do that? Yeah. Okay, perfect. It doesn't need to be now, of course, again, it's just adding more. But personally, I thought it was missing that you rely in large part on a distinct project, which normally is distinct, even though it will never diverge, obviously. So, same call for consensus. Sick observability is happy with the section above, all agreed, any disagreements? Maybe talk about rendering in stuff that's part of the development also happens in a different project, but you're part of that project with several people, go out. Release model versioning scheme. Everyone who's following the releases by Thanos will know that this is solid. So I would also suggest call for consensus. Sick observability is happy with the section above. All agreed, any disagreements? Very good. CICD status. Yeah, I know that they're running CI jobs and they're usually green. Matt, would you like to see something about regression testing in this section? Again, to go from sandbox to incubation, to be honest, I don't know that we would want to say full regression suites are part of the minimum criteria. I'm an apologist for the noise in the background. Yeah, so not personally, again, my question before was more just having been away for a number of weeks and not doing enough homework prior to this call. I mean, certainly if we want to articulate in that section, what is and is not there, that's fine, but as this is all in GitHub anyway, I don't want to block anything to get that question answered. Okay, so for everyone, obviously in this call, if you're happy with what is in the document, and if anything which was interesting in the discussion needs to go into the document, then now is the time to say so. I had one additional question about CI that I also had for Cortex, and I'm assuming it's the case, but other folks can replicate all of the CI results. I mean, everybody seems to be using that, but I haven't looked at that one specifically. Like if I would, again, I want to make forks of this and run it with my own GitHub actions and my own Circles CI account, I'm assuming that that's possible to do. It's not hidden away or buried in a private way. It was architected that that's not doable. It's definitely doable. One thing is that we test against, for example, Google Cloud buckets and AWS buckets, so you would have to actually set up like projects and buckets in those cloud providers. Other than that, you can either skip those tests or just copy paste the tests and run them. And the motivation for the question too, and maybe, again, optionally we could add it, but if there are, I, for example, we're running with Cortex, not Thanos today, or will be soon anyway in production, but I'm more than happy for projects that we're using in my setting to contribute test results back, or maybe I'm running some hardware or some storage that's different than what's tested upstream. If there were a mechanism for me to report those results back to the community, I would love to do so. And just in the spirit of accepting contributions, not just for code, but also for testing cycles, there could be lots of folks in the community that want to do that. So again, I don't think this is anything that needs to hold up moving into incubation, but as feedback for the project, if there's not a happy path to do that, consider it because I'm sure folks might contribute, right? You could find more issues. Yeah, I think it's doable because we have everything baked into Makefile, so which is just one Makefile command. However, yeah, some documentation on the details, how to configure with external object storage might be worth doing. Yeah, I've found in the past that complexity, the work is more on what do you do with all the results? And again, I don't want to put a barrier to entry to get into incubation to solve that problem. I think it's CNCF wide. If we could come up with some way to do that or see if other SIGs have already sorted this. By results, you mean like performance results or like just test results that they pass or? I more meant performance results, like for folks running Thanos at production, there's a various topologies and architectures and everything from, I don't want to take up a lot of time, but I would expect that the capacity of the community at large to contribute back for those willing to do so. For example, and again, this is about Cortex on Thanos, but as we roll to production with Cortex, we'll have a whole separate staging environment that replicates our production environment running ahead. So that might be a different set of workloads and stuff that I'll have test results for that I'd be happy to put back to the community. So yeah, but again, I don't want to, I've already taken up too much time because for this document, I'm happy with it. So call for consensus. Sick observability is happy with the section above. All agreed, very good. Licensing, it's Apache too. Sick observability is happy with the section above, all agreed, kind of obvious, yet wanted to make it explicit. Recommended operational models, that's also fine in my opinion, that section, but again, speak up. If not, call for consensus. Sick observability is happy with the section above. All agreed, any disagreements? Very good. Why is this different colored? Okay, so this Q&A with the project more or less, I read through it, I was happy. And I think this is also a copy and paste of something which we had before, correct? I'm sorry, my audio glitched and I had a distraction. On the previous one with how we operate it, I see that there's Jasonet. Is that the recommended way to deploy Thanos or is there helm or other mechanisms as well? Or are those kind of treated outside the Thanos project and are like, I could go look, I'm assuming there's a helm chart somewhere. Is that not sanctioned or maintained or? So yeah, right now, home charts are basically community-driven. And I think there are two or three people which are currently collaborating and basically working on to merge the home chart into Thanos community depot. None of the maintainers actually use home, so that's why we don't have home charts right now, basically maintainer support it. But we will shortly have one which isn't like a Thanos community organization. And yeah, people will be able to actually use basically community maintain helm repository. Also that is customized versions and yeah, a lot of stuff written by community. So just to add to this, like essentially we want to be focused on the project itself. And of course, as maintainers, we have kind of preferred way of deployment, but deployment model is not something that the project should recommend. It's rather what you already use on infrastructure, what you already have your best practice, right? Provilla series customized. We use JSON and sometimes operator because we have two Thanos operators as well from different kind of people. So I want to kind of make sure we are on the same page that we are trying to move this outside into community, however, help and make sure it has best practices, but we want to collaborate with the community instead of having this very controlled by us. Okay, thank you. That's also consistent with a bunch of other projects that we've been finding, right? I don't have a huge personal love for home myself, but sorry Richard. Yeah, no worries, no worries. Just looking at the time we specified 40 minutes for the due diligence. We are beyond that time, but we are almost done with the document. So I would suggest we try, oh, I see so whoever put- Let's finish it. So the questionnaire, I retreat, I was happy with what was in there. Are there any concerns, any comments regarding this questionnaire beyond what Michael put in? Or not. And I know it's quite a chunk. We can also take some time to read it if people want to, but then these just states so we know. Else personally, I was happy with the whole thing. Okay. I don't know if Michael is on the call. Else, should we as to say to require a few more vanity stats or are we happy with the document as is? I think I can like add the darker poles or like, yeah, stuff like that. Because we don't have anything else, I think. No, no, I mean, Michael just wanted to add some kind of example customer sizes of the deployments. So I think this is something we can definitely do, but yeah, I don't think we require that from Cortex as well, but yeah. Don't think so. Okay, so I mean, stuff like, yeah, I mean, I don't see answering this with more stuff as a hard requirement within the scope of this call of agreeing with the document, but more stats are always nice, even if it's just for sheer vanity. So yeah, I need to copy this. Same call for consensus. Yeah, I concur with the previous, this is great, I think, and I'm happy with it. In its current state. Okay. Good, good, good, good. So call for consensus on that section. Stick of solubility is happy with the section at all. All agreed. Are there any that aren't, I know that folks don't have to speak and then things, but we really do want to be inclusive of divergent views. Like if there are folks that have concerns, please like we heartily welcome this hand or alternate opinions and views, so. Most of the time people don't speak up, which is kind of a pity because I really would like people to speak up more, but just don't be surprised if we don't get a lot of replies. So final call of consensus for this document. Stick of solubility accepts and is happy with the document as a whole. All agreed, any disagreements? I'd like to extend like, you know, again, a warm thank you for the authors that put all this together. You know, the level of detail is great. You know, it's, I think it's a good document and it's a good example, like the Cortex one was on this. And again, I think having this at this phase will make subsequent phases easier. I'm fine having a slightly higher bar than some of the other six on some of this because this feels to me like someone new to the project that says, hey, what's Thanos can actually start at this diligence document and kind of very quickly have a good, well-rounded understanding to be a contributor, which to me is one of the most important things about this and this TOC can also spend, you know, a not crazy amount of time to quickly be able to evaluate it and have a starting point that's important. So thanks. Thank you. Yep, I can also, I can only second this. I like the level of detail and care which has been put into those documents and technical review. So very good. We... Also, before we're done, just one of the show time, if you haven't actually put your attendance in, oh, I see most people have added it in, everyone. I got Carlson to do it in the back there. Thanks, Boban. I didn't add company, so please add your companies and affiliations. Yep, good point. So do we have two minutes for last topic on the agenda? Yes. All right, so let's just quickly mention what this is about and we can expand on this and talk more like in two weeks. Essentially the idea, if you click the link is on the mailing list and the idea is that we are missing the connection between the kind of metric data into some kind of analytic use case, right? So that we have lots of nice solutions for gathering them, for storing long-term storage in long-term storage, you know, those metrics. We have Cortex Thanos, other projects and we want to make use, analytic use of this data, right? And maybe switch from more monitoring, real-time monitoring, alerting use cases to also include maybe some OLAP and, you know, business analytics and intelligence. I think there is a missing link and it's super hard for those people, for data engineers to gather efficiently and discover the data that we have in metrics. So as kind of from a user ecosystem, I would love to explore, you know, possibilities to integrate this data into analytic forms. So I'm looking for you to collaborate for us together. So if you have any analytic system that you are using right now that you would love to connect with your metric data, for example, of any other source, just please mention on the, for example, on the mailing list or on the ticket. Also join us on Slack, CNCF analytics. I just created kind of the channel. We kind of are starting to have some technical discussions. What API should look like? What are the requirements? And I think we can talk more about that in the next Seq Observability Code, but there are some nice questions that we are looking for to answer how we can move that forward. And second is like, is it really Seq Observability concept? I think given the charter that we all kind of contributed to, that's certainly in scope for this exact activities. I look forward to talking in the future, but we're running a number of analytics and a lot back ends. And we're doing specifically this. We've started and we have some plans. In our case, it's Snowflake, MySQL and Clickhouse. But yeah, I know we're short on time now, but does anyone else have an opinion? I mean, I think, I can double check, but I think we called out this scenario and this topic in the charter explicitly as in scope. Yeah, I think so, yes. But we still had some concerns on the emailing piece, but I think we stated that. Yeah, I'll follow up there. I would not be too concerned with like territorial claims. It's just like, if people are interested in working on something, it's just like working on who cares what, like if there is some other thing. That's not a concern. There should be some synchronization, but I agree that I don't see a concern about us doing this. On the contrary, I have been yapping about this for like five years now. So I would love to see a push in this direction. Amazing. So let's have that on the next meeting agenda. And in free time, please offline, add more feedback if you have any. And Matt, good point about scope. Yeah, I actually- This is relevant broadly as well. Yeah, I mean. Okay, okay. I actually have a hard cut. So any other topics or are we done? If not, see you on the mailing list on Slack and in two weeks. Thank you. Thank you, Richie, for leaving this. Oh, it was super efficient. German efficiency. See you. Bye-bye. Cheers. Bye-bye. Thank you, bye.