 Hello, welcome. Welcome. I hope everyone had a great summer folks. Just a moment to join Before we get started while we're waiting. I'll just say welcome. This is the sick observability First meeting of the fall after taking August off as many people were on holiday and coupon happened I'll remind everyone as we're supposed to do that. This is a CNC event and the Code of contact applies obviously, so it's never been an issue before but it's always worth saying Bear that in mind with any content in chat and his Richard here as well put the document in chat Please do sign in and if there are any additional agenda items. I think we still probably have some time Either your Richard Yeah, sorry Whatever reasons soon keeps crashing So you should give me Okay How do you do the interest, but I couldn't see anything I have a clash for the last 30 minutes Unfortunately So Let's walk through the agenda. We didn't have any TLC updates Obviously, of course, we didn't have any any Any calls and all just what happened is the TLC voted on both Cortex and Fedos and actually Finish voting and so both are now in tube any projects as of to come to come will use as the Lonesome for both of those And that's basically it for TLC. There was one other update. I spoke with Amy today It looks like we have 11 binding votes on the election for our tech lead so someone on the call deserves a A round of a welcome and congratulations and all of that I believe they should be putting out some communication in a bit. I know that There's one TLC member who is We'll be leaving the TLC for four reasons of a personal nature but I think with 11 binding votes we could probably say that it's It's done. So congratulations Yeah, thanks, thanks, but let's let's let's wait for official blog post and think but Cool, so I can I can start with with next topic straight away We have like just want to kind of get Get back In kind of the major thing we did like before The break in August we are talking about analytics APIs for observability signals and we started with like Prometheus and how to get Some analytic use cases on top of that. So they're didn't I mean nothing major changed since then however, we took some action on the red hat side and we started the short like POC project for Certain APIs and like the the report is called Opsletics on the Thanos community. Please join us and contribute the idea is to have Just just a proxy that will Essentially understand the Prometheus and Thanos Export APIs, which is remote read and store API. They're really really similar and it will expose a certain up analytic as APIs and right now We focus on pocket file and we also want you for you want to add an Apache arrow flight, which is a JRPC Endpoints so this is the plan, but we essentially it won't be no super efficient from the start but it will do the work and enable folks to just Use metric data much more efficiently in those systems So if you want to have I don't know if you want to add another end point to folk to to you know support maybe other obstructive signals or add any Anatic APIs. Yeah, feel free to propose that in the github issue on the PR. So We can we can do conduct and at the same time Kind of reminder. We are Going through use cases And this is the dog like this sick observation data analytic use cases high-level use cases What else we can do in our community in our? projects in terms of what is missing essentially For analytic use cases. So I think we can get back to this at some point for the next meeting, but please Yeah, let's reread this on its own time and and add if anything is missing I think we are in the middle of this doc or something So from the next meeting we can we can continue in that. So I think that's a status that was happening on our side The anyone has any questions? I think one question. It's very do you target this API? You mean the obstetrics, thank you Yeah, I Mean where do I want to implement the CPI it's essentially a Just a proxy just a glue between Something that will share an endpoint of Apache flight, for example, or it will just a bad job that will transform Whatever data you want into pocket files. So this is kind of two modes that we are aiming for one is a bad job and one is like Yeah, service stateless service that will transform the data from one format to another Hopefully without not much overhead. So that's the plan current plan Given that I'm not into the session I can't let anyone in I think maybe you should be seeing a Not sing and admit I'm actually not the owner of the meeting and zoom land Maybe in the future I can be Is it Actually a host. Yeah, I'm not a host or co-host. Is it No, no, no, that's me and I cannot yeah, I don't have any rights and You just need to log it right So maybe he's using wrong link Yeah, I'll put the link in the doctor. I'm sorry for the Maybe they are not Yeah Yeah, it might be the security thing to avoid zoom bombing where you have to be authenticated somehow That was like it. I was also forced to look in into my browser Before like within the browser was forced to look into soon again Before being allowed to join so I wrote this into the Hopefully they work Sorry for the direction Yeah, I think we finished analytics one. Let's get back to this on the next Next discussion. So just so we're just so we're clear started with other Hello, blue. I missed a question for me Is the idea here that that this is something there's an API that you're implementing in in Thanos to to help export Interability data or is that just expedient to create it there and we really think that this is something that would live That would use either, you know, some That would work on on Prometheus itself versus something that requires Thanos No, no, no, so this is This is only well, we needed that for the Thanos community But because Thanos has really similar API to Prometheus just jrpc This is also working against against Thanos, right? And for now like with everything else like we we talked About use cases and you know what API's are the most important But we actually have a real use cases on inside red hat to use observability data for Parkett Well for something that consume Parkett files So we went ahead and create this helper and this will and this actually a good first step because then if we see that There is like super high usage of it and and very And maybe there are some blockers or something that means enhancement Then it will be a good step to forward discussion on, you know, maybe Prometheus should have that by default on On their side. So, you know, this is actually some some actionable thing that we chosen to do So, yeah, Parkett files is first step, but yeah, everyone is welcome to add their own Just to have something working for us that thing Just want to follow up question to that can this also be used To export so you you're taking data from a remote read Can one of the exporters be a remote right and that could be also a tool to transfer Data between the various systems What does the thing like we are at least for for let's say and that's a good point like was the scope of this of this tool this tool We want to make sure that you can use the data That are already ingested by Prometheus Thanos or even cortex like can can can can easily export this API We want to make sure that you can use this analytic systems like You know anything kind of all up related like the Parkett file Anything that comes in Parkett file Apache arrow things like that So even like Python notebooks lead Jupiter things like that All of them can consume and work on this data while still have data in the same place because remote right is all about replication and Well, you there are already tools that kind of allows replicating in different databases But we want to kind of allow using the data that already are consumed and stored in one place and Try to avoid replicating them in many thousands of you know Other locations that are maybe You know have different APIs. That's why this is later Helper for like a pooling method like pulling only the data that you want for for analytic purposes That's why remote right Doesn't fit here. Yeah, but you can you can totally do do this do You know have Prometheus and have remote right and push the data and replicate all you need for analytic use cases There's some analytic backend But you know, that's totally separate story. I guess but that's definitely doable. Yeah Awesome. Any other Thought question No, I hope to have some better updates in a couple of weeks within our own team About two weeks ago. We started working on some of the scenarios that are captured in the doc that we had talked about before around machine learning and model training And and we've since deployed cortex in our case And we're looking at various ways to do the same. So I'll have a look at what you've posted from the Thanos community and Yeah, perhaps a more concrete results in a couple weeks But it's it's been interesting to see a team that's used to having to ingest data and then build an entire database Just to hold things and they're there They're looking at Prometheus and the data model for the first time and and they're finding that they might Be able to prune a lot of work just to do to have Prometheus is already so optimized for types here's data So results so far positive, but we still have this core need to be able to bulk bulk read historical data Exactly and the main blocker is that from use remote read and those even remote write APIs are very hidden There's not much documentation for it. So if we enable such kind of show how easy it is This will enable lots of people to to stop you know writing their own tools from scratch and then having and just reusing the data that is available to you site and For particularly your use case like make sure to also poke cortex to enable remote read or We can talk how to do it internally because the data model we have really this have the same So it would be amazing if we can reuse exactly the same the same tool and Yeah, would be up for that for sure awesome, right great on the next one who who who had put in the Growing the observability community with CNCG Hi Matt. I'm Piyush and I work primarily as a DevOps engineer with dream 11 here in India Hi I've been trying to build a CNC of community around here and As I'm a big follower of observability myself I have been looking for ways in which I can contribute back to the community and There are a few things that I Observe during the coupon as well as With the new platform that we have put in place for the hosting since if communities just for everyone's update and context CNCF is moving away from meetups and Is planning on self hosting the entire community platform along with baby And that's what the concept of cloud native community group is So the first suggestion that I had was like so with CNCG the first thing that we have is We have divided the whole community into different chapters according to locations right now, right? And that's just a horizontal distribution across the globe I was hoping maybe we could have a vertical chapter sort of for observability and With baby you can add sub chapters on to it So basically if you add a single event to a single let's say appearing chapter it shows up in all your sub chapters Okay, so if we could have a vertical chapter around there and then we can Make all of these locations of chapters as a location chapters as sub chapters of that We could have a larger community who can be you know engaged in a single event in a single observability event and Like I was really hoping that we could engage the community better in this way The second thing that I had was we have a bunch of keynotes that are available for free for everyone within the community itself I'm really sorry within the community itself and as for Kubecon itself. We had keynote passes and everything So it might be a good idea if we could have these keynotes Available on that community channel itself because that makes Engaging the people working with the community within the community chapter itself The third thing that I had was It's just a general suggestion that we post these videos or these talks or any useful links that we have resources Within that community itself and you can do a bunch of different things. Yeah, you can have Newsletters and everything over there Within the single chapter. So people generally have a common place where they want to talk to Everyone interested in observability, right? And you can have Resources around there and the last thing that I had was basically so What I found when I was Trying to contribute to the observability community and SIG is We don't have a lot of documentation at the moment around it right and it might really be a good idea to have some sort of documentation so that People know how they can contribute to the observability community on a broader scale and not just for project specific like contributing to the community as a whole right so, yeah, these are just a few things that I wanted to bounce off of everyone and Yeah, please anything that I can help out with Great. Thanks for Thanks for for covering that I put a couple links in the meeting document in terms of the This the videos from the from our SIG meetings now and moving forward the CNCF records And post all the the videos in a playlist under the CNCF YouTube channel so if there are our Community groups or other places you want to link to that that would be the authoritative source at least from the SIG mechanisms In addition for some of the documents that we review moving forward I had hoped to have the link for Mamie today. I'm told it's been provisioned But we have a Google Drive as well for Google documents moving forward That we can have for SIG observabilities use I must confess that I had somehow missed the blog on the community groups. So I'm not an authority or anything Does anyone else have Feedback or I think we also talked about some entry point For for like observables topics so that that that once finished we can we can definitely point to that And my question overall to the overall thing like it's not amazing, but what's actually actionable Is here from from from this Sick, should we sign up for for sampling should we announce the keynotes or have ability for keynotes look for presenters? How we can help essentially So what I was essentially looking for is the current limitation that we have on community groups is their location based, right? So you have SIGs for local chapters, but you don't have them on a broader level So let's say for instance if you are in Mumbai, right? You wouldn't probably join some chapter in New York or San Francisco unless until You have some very special interest there, right? Or you won't have visibility onto it. That's the primary point, right? So if you could have a chapter that connects to a People based on the type of interest they have rather than the location, right? That brings the whole observability community together in that sense Okay Second thing that it could be a very good place to start for everyone because like the general trend that I've seen at least with the startups in India is with The entire COVID pandemic hitting and everything, right? Everyone is just trying to save cloud costs at the moment and they're trying to cut down on cost whichever way they can and one area in which they lack the efforts of They don't have a good starting point or something is observability. So it is predominantly our partners that we have New Relay, Candidate Organ, everyone, right? But they still don't have enough community around it. They don't have enough contact around So they can work with open telemetry or other tools like that, right? They don't have they just don't have that at the moment. So having a broader community might also help with that Sounds good. Did you I mean, what other Sikh groups are thinking about that? I haven't talked to I haven't attended any other meetings as such And I'm not really sure about that. Actually observability is an area that intrigues me a lot And I have been looking towards it for the past couple of years. So that's why I figured it might be a good place to start Yeah, so but you said it's a limitation of platform, right? No, no, no, so the limitation is currently the way we have structured it Yeah, so that puts a limitation because it is currently location based and not interest based Please if anyone has any questions anything that I can help out Yeah, I think No, I think as we said, it might be helpful if What was being asked for or what the actionable next steps were captured Either in a document or or ideally in a github issue and the sake of durability Github repo and then we can You know then as a community we can have some some feedback I think I think many of the things That you're mentioning We do have Some coverage for in the sick itself and and the sick exists to help with many of those goals No, but but what what you said essentially the platform makes sense also like I think we can I don't know if you're asking for this but Looks like you are looking for some people who will organize those groups or smith apps across across all the locations Right around if I can get any help with that that would be great But even if not like at least we got to start somewhere. So Exactly, we can definitely help with that. We can we can just announce it and find, you know, maybe some topics and And speakers because kind of that's that's what matters. Yeah, we would love to but Regarding so definitely let's let's let's cover this action item. Yeah Yeah And in terms of, you know, the the I don't know it's a parochial but the local the the local aspect of it, you know, like I think time zone makes sense But at least in at least in North America or for some outside North America too, you know, um with the current state of affairs You know, I'm not sure what what it what it means necessarily In practice because at least, you know, most of the local meetups and such like that are in person are at least aren't happening Until things so bevy provides just that Initially that was being held over zoom Some of the virtual meetups that were going around I think were being held over zoom and others but bevy provides just that great Is there anything else on this topic or should we move along? I don't think I have anything else anyone Is there anything that you can help out with? Okay, um, I think Ganesh Extending from okay over time Yep. Hi everyone. I'm Ganesh. So I'm currently working on uh extending promcule starting with a very specific case of Extending the current top k function. So before I talk about what it is. Let me explain what the current problem is Uh, so uh prometheus just allows a query and a query range and query range is just a syntactic sugar Or query where it runs the same query or multiple timestamps and returns individual results for those individual timestamps So if you run a top k query in the query range Uh, and if you have a top k of let's say five you it's possible you get much more than five Time series as a result for the query range because at every evaluation There might be different time series But uh the use case that i'm so trying to solve is you might want to look at Some aggregation or the past few hours or past some time and just pick Top few metrics. Let's say five metrics out of all those aggregations and just plot five metrics for that entire range Instead of getting thousands of metrics for every evaluation timestamp Currently that's not easily solvable with current promcurel or current apis Uh, because in the query range you cannot use the start and end or other parameters Of the query range and promcurel doesn't allow any such uh extensions So currently i'm trying to solve this particular problem I'm currently working on a design doc and exploring different paths to solve the two Directions that are possible are extending the promcurel itself Which uh sounds more promising which might even solve more problems than just extending top k and the other option is Introducing a new api which doesn't have restrictions like the query range currently, but that's uh Not so Good solution when it comes to extended promcurel So currently i'm focusing on extending promcurel So if anyone has more ideas on how you want to extend promcurel and more use cases You can Reach me out on slack. My name is Ganesh Vanakar Other than that, I don't have much to share in terms of solutions I just wanted to put a point that i'm looking at extending promcurel and would like some inputs if you have Um, should we open some issue just on promcurel just to cover some ideas as well maybe and just to Maybe attract people to to to put their ideas there. Maybe tweet about that. Um, so you're not pumped with messages Yeah, so I just started working on this thing yesterday. So that's the plan. I will be opening a GitHub issue and Maybe different issues for specific use cases to divide the conversation Because this is like very Kind of controversial topic like we usually didn't allow to extend of extensions of promcurel But I think it's super great idea that we at some point Want to move this restriction And yes, please help Ganesh to gather more essentially requirements For the potential solution here Yep, requirements are the main things And and also if you are also I don't kind of confused with the top k or you are particularly Um, not happy with this also rich rich Ganesh and and this issue. I guess just to give your opinion Since we are fixing it. That's a good moment to give your feedback Yeah, yeah, I want to be be careful though that um, you know this really is something for the primate theist community to Provide feedback on and has governance over in terms of a promcurel but you can certainly surface it here as a Call for You know, if anybody else is interested in it or such. So thank you for bringing it To this thing. There's no other feedback or comments on this one. Um, let's move on The next one's mine. I actually could do this in about 30 seconds. It's really just a mask I put some details there, but we had talked back in june or july And in in some of the charter discussions around You know providing something from the sig that a new developer that's new to the open source Space or new to the cncf stack could have quickly working on a you know in a local scenario Or on a small cluster scenario just to If it were coronet is hosted Just to to have a starting point. So I'll create a github issue to track this, but I wanted to surface it here. I have found some various docker compose things Some of the folks at we've works have have some of this as well so but before diving in and making a net new thing or You know, if there's prior art or if folks have found other things of this type useful Please feel free to either put them in the dock and I'll roll it into a github issue Yeah, have you have you seen much the katakoda tutorials? I'm sure the what? Katakoda tutorial platform Like I haven't actually in the dock. So this is like particular course for tannos we are working on And this is a platform that literally gives you opportunity to just play around those things and play, you know, kind of Just run tutorials across different Components like right now it's about tannos, but you can actually there are already tutorials about kubernetes Prom to use other projects And it's essentially works in our browser. So maybe this is what we Want to use it's kind of, you know open or well free to use for open source projects Um, so it's kind of much much more ready than than just trying to set up the Yeah, the airport Sorry airplane may have left the motor Something like that. So anyway, I will add that to the issue Oh, yeah, and I have um, I've found uh, again some like docker compose files that have been particularly useful That are a good starting point that that have you know a bunch of these tools put together But it's a bit of a patchwork now. So I was curious if Having something a little more curated by the sick for new contributors to the space might make sense So maybe we could do this on one of these other platforms or I don't want to take up too much time though If there's no other feedback I think that I I want to add something No, I am Sergio. I am from Guatemala Here in Guatemala. We are I am working with my students in the operating system course So I want to integrate some things about observability I have a some repo that I can share with you or maybe I can help in your goal to create some Kind of small projects or examples to use docker compose and that things I think that maybe I can join forces with you with the sick. So I I think that I can share here the my repo It's it's like a kind of well, I am trying to create some space for the students in the CNCF ecosystem So that's my main idea um I try to add some kind of samples there that I have some examples to implement jagged in an easy way um Right now we are trying to experiment using a linker g But in complement using jagged I can compare and optimize A kind of project for my students. So Maybe I can help with that. I have some experience creating Catacoda scenarios So maybe I can help in that way Yeah, that's that's great. Um on the linker d subject That's what my team runs in production as well. Uh, and we're actually We're finding that it's great to onboard people to CNCF observability stack In general because the the linker d proxies can generate trace headers As well as time series metrics In prometheus format. So for example on the on the tracing side Even for services that do not integrate yet with a tracing library that emits b3 headers We can still kind of have bookends For for for them. Uh, they can that can go into an overall observability story for a whole series of Interconnected microservices Yeah, I have some examples using the open telemetry library too So I can integrate that I am trying to create some very simple examples for the students because it's really hard for for them because it's a new technology And it's like a totally wow. What is this thing? And Yeah, I think that I can help in that way Great. Thanks um So, uh, I guess the the last thing um, I we had on the agenda was just update your discussion regarding existing tasks that are already As marked as issues. I don't want to do necessarily an issue walk because That might be a little tortured But is there anything folks wanted to discuss that they're working on or that they want to propose? Or that they want to just Get some Get some visibility on from the sake Hey, this is actually related to the previous question We have a brand new tool. It's not out of Beta yet, but it's a cli tool to actually bring up grafana From atheists, etc. And they come with a helmet shot as well. I'll post it on here It's called tabs and right now it integrates Time is scared to be as the back end still but we welcome contributions from other Backends as well. Uh, so kind of related to both of these topics Oh, that's great. Yes, please do. Uh, please do link it and um I'm happy to have a look That's great Maybe I can say something I'm not sure if everybody knows each other here. Um, it's the first time I'm joining The meeting Simone is talking so I'm actually on behalf of ericsson here Welcome Thanks Thanks, man. The SIG is quite new actually. I think we're all still getting to know each other So, uh, you're in good company. Yeah, I noticed the Yeah, so I'm working in the area of trouble shooting and observability um, we are going through well, if you have been Reading the news with all the ideas in 5g. So we are going through a major Transformation in the in how we build these networks And I work in the run in the radio access part So we are basically changing how we build the system and how The deployments are going to look like in the future as well So the the let's say the 5g is not going to be the usual Operator idea that we have with this antenna the mess outside It can they are going to be much more like smaller and complex And one part of Building the systems is in the trouble shooting and observability area. So we are moving from a more system softer Type of tracing observability idea to a cloud native And one of the ideas that I was trying to bring here or discuss more It was also in the in terms of White papers and also having Bringing this this discussion in terms of the 5g also this big I'm not sure I had the impression that you guys are more focused on the tools and some things that you are building and improving right now I would say that's maybe I'm in my daily life I'm like you but see here. I probably a layer above Maybe trying to get a bigger picture and trying to how we promote this the observability In different areas and for different purposes as well, that's not only That people don't only think about Which Yeah, certain use cases that are associated with observability right now Um, that's great. Actually, that's I'm putting a link in the chat to the charter that we recently Settled on a few months ago And and scenarios like that are specifically in scope for this sake You know how people are using observability tools and data Methods on on how to to leverage them and how to make them actionable or useful for various domains and verticals That's all that's all in scope And so we have a lot of latitude as a sig about how we spend time and and what we might want to either welcome members to have Talks or webinars and orchestrate that Or to provide a forum to get feedback on on ideas and or approaches in that space so Welcome to the sig and and We welcome all Yeah, I have a look at the charter, but but no that that's in that's in scope. Okay. Thanks I think that's all we have on the agenda. Is there is there anything else folks want to discuss in the last minute or so? Or should we return a bit of time to people's day? Well, I guess that's a wrap then happy September everyone and I guess we'll see you all in two weeks Yeah, see you. Thank you Thanks Okay, bye. Bye Bye