 Oh, hello, let me get my projector feed. Hi, Chris. Oh, hi there. The rest of you, I saw it 20 minutes ago. Actually, no, Tristan just joined. I saw you, Tyler. Hello. Okay, join zoom. Welcome to January 9th. So I am interested in conversation. Oh, stop that. So cool jumps joined. Here is a meeting doc. So last week we were talking about and then throughout the getter throughout the week we were talking about. We had this naming question and we sort of brainstormed a week ago and left with last week's notes have sort of few topics, potential names that were listed. I thought about it and thought about it and ended up liking accumulator the best. So I was just going to come back to this group and say, what do you all think about if we name the former thing formerly as the meter implement sorry, I'm not the bachelor. This is the meter implementation. Yes, yeah, I get myself confused. So I like the accumulator the most. I had proposed differentiator and the notion being that the meter implementations job is to compute all the change that happens over a period of time. And in some sense that is a differentiator, but it's also an accumulator. So we're accumulating all this, the sum of differences over a period of time. That's the way I think of it now. If anyone wants to propose another name or fight for something else, I will, I will listen. I don't, I don't want to be, I don't, I don't know. I think accumulator sort of most is the most conventional choice that we might make. In any case, the point of that discussion was to get to this metrics SDK, SDK specification, which has been an ongoing PR in the spec repo. My goal is to update that with this renaming and then push forward on that. Again, it'll happen next week. So if anyone else has anything to say, potentially we can talk about it, but otherwise I will propose that we move forward with accumulator and you'll see that we're showing up in a document next week. The topic, which I thought was most interesting came up both in the Gitter earlier this week with Tristan leading the charge and then also, also Yuri ended up asking the same type of question. And John contributed some of this to the conversation as well. He answered the best that I could in this, in the Gitter last week on the fourth here. So let's talk about this because it didn't really end. This conversation didn't end. It kind of was left open. Tristan, you raised it first, but I felt like I was prepared for the question because we had discussed it in our metrics like working group in August. We can go looking, I have gone and looked at other metrics APIs. Many of them are kind of concrete libraries. They're not facades or there's no API STK separation there. So those ones don't necessarily give us much to work with. But there are some out there. Micrometer is the one that comes to mind, which is essentially a facade API that lets you bind to your metric system potentially to more than one of them, much like open telemetry is trying to do here. But I felt when we looked at those that they were a very behavioral API rather than a semantic API. So what I mean to say is that the micrometer API, for example, had an API call which was something like New Histogram or something like New Summary. And those are very concrete types of metric behavior that we know about. We know what a histogram is. We know what summary is. And then as you went forward and looked at the open census library, they had sort of struck a balance between what is a performance oriented technique where you're almost surely going to be pre-aggregating. You have a counter and a gauge. Countries are really easy to aggregate by computing and some gauges are really easy to pre-aggregate by computing a last value. And then there was this leftover thing called raw statistics. And that was when I got involved in, I'm guessing it was around June or July, where the current spec for open telemetry was more or less copied from open census. And so it had a counter, it had a gauge, it had raw statistics. And my first reaction to that when I started working on this was, well, I really want to be able to do label sets. I know that that's a source of performance loss a lot of the time. I also think it's a good programming sort of technique to improve the readability of your code, which is to sort of think in terms of label sets. And then this action of binding, which is when you take a label set and bind it with an instrument, what you're able to do then is you're able to get this really optimized code path because you've sort of calculated a lookup inside the SDK, which says for this label set and for this instrument, this counter, let's say, I'm going to have a record in memory and I can just do an atomic increment. That was where we were in July or so. And I wanted to say essentially bring the benefit of binding or of having your label sets be pre-computed to this raw statistic field. And the outcome of that was that we created this measure metric instrument. So it's now it's an instrument that has a record verb rather than having to add or a set. It's a record verb. But of course, the still the exercise was to create an API SDK separation and very few people had done this. And so it got us into like talking about what is the meaning of your instrument? What is it here for? This is why my answer to the question was really about it's about semantics. So every metric instrument now has some sort of very similar structure. An event records a number with some labels, which are called label set, and it has a description, which was when it was defined. And when I saw the questions coming up from Tristan, from Yuri, from John this past week, it was really sort of regurgitating this conversation that happened in the summer between me and Bogdan and some of the other people in the working group. At that time, Chris may have been there, which was basically saying we think that we don't want to go overboard and say, hey, everything is a metric. All metrics are the same because when you read the code, you want to sort of know what's going on. You don't want to just see everything as a record statement because truly everything could be a record statement, but it's sort of like a loss of information. If every programmer turns every metric into a record statement, regardless of whether it's a counter or a gauge or a measure. So I'm trying to say what I'm trying to say is that there's a valid observation here, which is that when you talk about API SDK separation, all metrics could be the same because the SDK can do anything. That's kind of our goal. So why do we care whether it's a counter or a gauge? Because you can turn them all into whatever you want. So the answer I think is we left that in. We left there are three types of metric. They have three different verbs. You will use them according to what your verb is most naturally. If you are thinking in terms of I'm measuring a distribution, you should use the measure and call record. If you're thinking in terms of I'm computing a sum or I'm computing a rate, you should use the counter. If you're thinking in terms of I've got this value that I'm computing and I want to set it, then you should use a gauge. That's the guidance. And ultimately, as far as API design goes, we want to make sure that the code is readable. And so my belief is the reason why we have three types of metric is to make the code readable. The SDK can do anything it wants. You can compute a distribution from a counter. You can compute a sum from a gauge. These are just configurations though. Anyway, I've talked and talked on that. I don't know if anybody else would like to join this discussion. What do you think? So I'll just chime in. I was really just kind of brainstorming and riffing on discussions that Tyler and I had in person. I don't, I actually think that what you have is pretty much exactly right. I would think, I think probably it would be in the future, not for 1.0, but we should think about some higher level abstractions like timers or things like that. Timing is a really excellent and special case. And I admit that. So I'm even willing to like try and bring it up now, like file an issue. And so, but the way I see it is that, and again, I'm more kind of more familiar with dog, dog stats. D we use that. We've used it quite a bit. The stats D library from the very beginning of the Etsy stats D client had the notion of a histogram and a timing. And they're behind the scenes and most of the systems that use that data, they are the same. They are just distributions. Whether you call a district histogram function to record a number or whether you call a timing function to record microseconds or milliseconds. It doesn't actually matter. It's all the same. And so in our sort of efforts to kind of pare down and, and, and get to the minimum sort of core API features. We, we kind of took that recognition into, into account and made there just be a measure metric, which would be useful for recording timings. But I do see timing as being a very special case. And there's two things here. One is that there, there's a common abstraction in code, which is like a timer, like a stopwatch where you say start. It computes the time and you say stop and record or something like that. That's a helper that we might want to have. But there's also like an API, which is essentially a metric instrument, which you would call a timing instrument, maybe, which, which you record measurements of time, which are durations. And in order to get that really kind of safe to use, you want to make sure that there's a type which says what the units are. It's not good enough to say it's just a floating point number. And you should remember to attach the milliseconds unit. I don't think that's good enough. I think because most programming languages have a built-in notion of time, they have different very variable types in play, like Go uses nanosecond durations. Java uses milliseconds very frequently. So there's a convention in each language for what your standard unit of time is. We should follow that, that standard and that whatever the sort of convention for time is, I think there should be a wrapper or a utility function or a helper, which says timing instrument. I think it's going to happen, but it hasn't felt like it needed to be discussed. Although you're right to point it out, John. Yeah, I don't think, as I said, I don't think it needs to be done before 1.0. I don't think the only, the only other thing that Tyler and I kind of riffed on a bunch in person was this idea of having a way to communicate the semantics beyond just counter gauge measure, like communicate some semantics between the instrumenter and the SDK. So that there is some, some handle of some sort beyond just the three types that could be used to choose intelligent defaults in the SDK for how it should be aggregated. And I don't have any good proposals or anything like that. It was just something that we kind of threw around. So there is a history here. The current spec says that you can put your default keys into the declaration to the definition of the metric instrument. You can say with keys and that's, you know, the current go SDK provides two batchers. We'll call them, we'll call them integrators in the future. That one is that uses the default keys, which is very much like what Prometheus would do. And one just uses all the keys, which is very much like what dog stats would, dog stats D would do. So having the ability for the programmer in the code to define the default keys seems like one step in the direction that you're asking. In the past, I myself proposed something that was, was rejected, but it may also sort of be like what you're looking for. So what I did propose in a very early O tap was number four, I believe was that the programmer could also specify the aggregations that they want. So that you could declare something which is a counter, but say that I actually want to, I want to count, I want to monitor the distribution of count values. In other words, I want my code to say add because I'm thinking of it as a counter, but I want to compute a distribution because it's useful to me to know how big the increments are, but on average, right? And that's not something that we can do in the, you can't declare that I want to counter with a distribution aggregator in the code right now. That's something that we can configure in the SDK. We can have a configuration that says for this metric, which is a counter, please compute a distribution. We can do that. Or you know, min max some count, my counter. Yeah. So this seems like something that we could add to the API after like after 1.0. I agree there too. Yeah. So, so if we all agree that it would be nice for the programmer to be able to declare, you know, which aggregations we could add that after the fact, I think as a hint, because the whole point of having an API SDK separation is like somebody is going to come along with it. Like just doesn't care about your hints and wants to do what they want to do. And so when I wrote attempt four, it was just an advisory thing, much like recommended keys or default keys. We're calling them now, you have recommended or default aggregation or more than one of them even at this point, we're almost configuring views. So then could you have like, please give me this, this aggregation with these dimensions. It's one step further. If you're listing aggregations and dimensions, you're basically that's a view at that point. And for whatever reason, I think I haven't thought about this too much, but the open census library did use a separate library to configure views. In other words, it's not at the call site where you declare the metric instrument that you declare the views. It's some other third party or third or some other function that you call to go through and register views. So, so that's the competing kind of vision is that you have a separate way to do that. So they have, I mean, the reasoning is the naming essentially is one of the big reasons is you might be have a measurement, an instrument that is recording the latency of a request. And you might want a count of how many requests you're doing. So you have a view that is server request, request count, and you aggregate by a counter on the latency, the latency instrument, and then you have one that's server latency that just does a distribution aggregation on the, on the instrument. And so I think I like, I don't know, I don't like the idea of waiting until after 1.0 to figure out where aggregates go because right now I don't see how as a user, I would use open telemetry metrics and get what I want. Well, what does that mean? So beyond specifying whether you want a counter or a gauge or a measure, what do you need from this that you, the user think you should write in the code, not you, the user, when you operate your main function should configure. Why does it have to go in the code? I'm not saying it shouldn't. I'm not sure what you're thinking. Well, I'm thinking of as the, so it could go in the main function. Because I think what we're, what is the, what I believe is that it should go in the main function, the configuration for which aggregates and which views and which dimensions are being aggregated. That's configuration stuff. The question I see or I thought I heard maybe from you and John and Yuri was more along the lines of shouldn't we be able to say new metric instrument with these aggregates, with these views, that's an API then, not an SDK configuration. Yeah, no, for at least for myself, what I would like is it to be separate. Like I like how we create instruments right now and don't think aggregation should be defined within them. I mean, that's one of the reasons I had the question about counter-engage in the first place was because that was seen as defining the aggregation at the time of the instrument creation instead of later. Yeah, I hear you. I sort of like splitting hairs at this point, but the choice of a counter versus a gauge, in my opinion, is like that that's suggesting or recommending a default for the aggregation by virtue of the semantic nature of the thing. So I'm recommending that you use a sum aggregate because I call it a counter. I'm recommending that you use the last value aggregate because I call it a gauge, but the SDK can be configured. So for me, these last two bullets that I have my cursor on, the real question is, are you hoping in the future that we have a view API that's part of open telemetry, or are you hoping in the future that there's a configurable SDK that lets you have a YAML file that does this? In other words, is it a library of code that uses the links against open telemetry, or is it a YAML file that's put into the SDK? Probably a view API. Yeah. So since it's had a view API, and I, so. I don't know that it has to go in the open telemetry API and not the SDK. It's sort of a question of whether any other SDKs are expected to do this or not. Right. And I think it's, it can be done one at a time. Like you could put it in the SDK first and then we could create an API after the fact. I have looked at the open census view API, and it's pretty minimal, which is good. You know, like it doesn't, it doesn't have a crazy amount of flexibility. It's most, for the most part, you name the instrument, the dimensions, and the aggregation period. That's it. There's nothing like complicated queries. Like you're not feeding the result of one view into another, for example, which is what databases do. So I, so I'm, I'm not against an API, a proper API for views. And I would, I would probably suggest that we go with getting like there's a spec open issue about a CFP for metrics config, because I see lots of possibility. And then there's in the, there's some prototype issues filed in the go repo to kind of get the mechanism that we would need to get the configuration to work. But then I could imagine people still are always asking for it in that, in that ticket, I forget 381 or 382. It says the configuration should be a struct or a protobuf or something that we can reason about so that you could imagine the code writing it. So then you have a library of code, which generates a config struct, and then you configure your SDK. And then maybe in the future, we turn that library of code into a proper API so that more than one SDK could be configured using the same interface. Yeah, my only concern about having it in the API would be covered by that because I'm thinking of like instrumenting middlewares and libraries that I have that I want to be able to export to a user some default views that they probably want to subscribe to. I don't want to have to depend on the SDK and we're going to do that. Got it. So that's what OpenCensus was after. The GRPC library does come with a method that you call to configure all the views, and there's two versions of it. There's like the minimum, and there's the expensive version or something like that. Yeah, I like that model. I would definitely say yes. Let's look forward to a View API. I feel like we're not quite ready for it at the moment, but that's only because we haven't figured out, we haven't actually got any code to do that in the SDKs at the moment, much less config struct. So we're saying that like more contributors would be helpful here. And I mean, I'm working on this SDK spec to kind of bring it up to the point where the Go SDK is, but there's other things going on here. So if anyone feels like they could be contributing, I'm not sure what the best place is, but potentially writing a CFP or like an OTAP about a View API, or if I was to click through some of these things. In the Go repo, I'm thinking of these two issues here. I will eventually get to these myself, but there's a lot of other things happening. So these are the idea that we need to see some code that implements the views. And then we can talk about how to configure them. The one I was just referring to, this is the one I'm working on. This PR, this will have the renaming that we talk about and some diagrams added before the next time anyone needs to review it. Oh, it's worth mentioning that we've got one of these closed this week, metric observer instrument has been approved. So now it's time to write that into the spec. And that will need a ticket to get implemented in the SDK, at least the Go one that I'm using. So anyway, I'll file that after this meeting. I don't want to run through and talk about each of these, but this is a list of current kind of like protocol related issues for metrics. And then the one I just referred to bringing me back to where I want to be is 381, 382, sorry, 382, 381, 37, 391. These are all, actually I should say 381, 382. At the spec call this week, I talked about 381. I have a prototype about resources and scopes that make it possible to pull the label set out of metrics and put it into the scope. I'm going to share PR and an issue on this topic tomorrow. So I will not say any more of it about it right now. I was just trying to say that this 382 talks about the ways that we might configure metrics. Eventually views will come out of that. So like it says, and I think after this happens, we can talk about a view API. So there's a lot of kind of dependent pieces. I'm not sure if right here in this meeting is the right place to talk about sort of tactically about how anybody else can contribute. Maybe that's a good thing for Gitter. So I want to invite anybody who is itching to contribute to join Gitter and we can talk about pieces that are that can be broken off here to get more metrics. But for the most part, I've been owning a go system. And so I don't, and I actually, I think it'd be nice to do a roundup of the current state of metrics in each language. But I don't know that we have the right people to do that right now. Maybe that should make that my issue for the next week. I'm going to make an action item for myself. I know that Java has bogged in code that's been migrated along from OpenCensus. And I don't know how far it is from the current spec. I know that Python has a version that's somewhat behind the current spec. I know that JavaScript has done something and I'm not very familiar with it. And I know Tristan that you're working on or lying. That's all that I'm aware of as far as metrics implementations. I actually know C Joe came a few weeks ago and was interested in working on C sharp. Anyway. So this meeting is sort of both an office hours and a SIG. We've kind of talked about all the hot issues. I don't have any more pressing things I wanted to raise or talk about. Would anyone like to take the floor and talk or ask questions or discuss. I expect to be working on metrics more next week. This this week I was sort of sidetracked on scope question, which I'm excited about. I think it's a good step forward. And it's also nice to see label sets fall out of the metrics API. Without losing the optimization that they were there for. I also got one more little benefit out of that that I that I had hoped for. But I'll write that up this week before the end of the week. I'm going to update the attendees list. This is a big meeting. Here we go. Chris. I don't know who Brian is. If you want to speak up. Yeah. Brian. From New Relic. From New Relic. Yep. Great. Especially good to know who's who's which which is companies are getting their hands dirty here. So that's great. We have a lot of New Relic representation. Please to see that. I have now updated the attendees list. I still have nothing more to add to this meeting unless anyone wants to talk. That's just for reference. Brian's also the one working in the C plus plus. Oh, now I connected it. Got it. Thank you. Thank you. Thank you. And I've been seeing your peers then. Great. Nice to meet you, Brian. Sorry I didn't make that connection. I don't need to put company affiliations. It's mostly the ones from industry that matter to me. Just know your postmates and mostly the independence. It's the ones that are. Cool. All right, everybody. Last chance to have an office hour question. Josh, one thing I can think of that might be helpful for people coming to metrics that haven't been following the PR is posted. It's just a like a quick summary of. Like which aspects of the API you still want. Design feedback on. What I see a lot happening at least in like Python and Java is that people are commenting on metrics, but they're coming on the implementation. Right. And we're basically just taking it on faith that the go design is the one that we want. Can you. So where are these questions showing up? For me, they're showing up because I'm reviewing Layton's port of your 347. Okay. In which language. In Python. In Python. Oh, right. Layton. Okay. So can you be a little bit more concrete? So you said that there's sort of confusion over. The API or confusion over what the SDK should and shouldn't do. Yeah. So this is like a pretty vague question to begin with. So it's hard to be more concrete about it. I am. I want to know like which, which aspects of the API you still want comments on. And I mean, I know that this has not been like months in the making, but it's just hard to know. Yeah. It's my sense that like a very few people paid attention when we were doing this work, you know, Bowden paid a lot of attention. I was certainly paying attention. But as far as like everybody else, you, you paid attention. I know that. But, but it does seem like this sort of question, like the ones that Tristan raised in the Gitter or the one that Yuri put into my, my DM on Gitter where like maybe realized that not everyone has followed. And maybe that means that we should like all of it's up for debate at this point. And this thing I'm doing with scope, which I will share by the end of the week, actually was it is a significant change in the metrics API, but it's not like a substantial in the way that like it's not changing the mechanics. Really, it's just sort of pushing sort of sugar around to make it a little bit nicer or, or to put resources in place. So I think the answer is we still want comments on everything, but I feel like we've solidified quite a bit and I'm hoping not to change it much. I'm hoping not to change it at all. And at this point, all the questions for me are in the SDK and what can it do and how is it configured? Or is there a view API? So, and then I think it seems like the OTLP protocol has more or less stabilized. And I know that, that people are starting to ask, hey, can we send these metrics somewhere? So, unless somebody comes up with an urgent disagreement over the API, I think it's settled. But I took the first part of what you said to heart, like more documentation of where we are and what's what's happening is probably what's useful. So, and document this, I think is what is needed. I know that you've done a ton of work publicizing metrics already. Well, it's not enough. So that's future work. I'm aware that that that's needed. And then we have some content generation targets at light step just to like promote like open open telemetry. And so I'm going to be writing about metrics more in the coming quarter for sure. I suspect what's going to happen. I mean, I think you have this suspicion too, is that right now it's hard to get anybody to look at it and there are only maybe like a dozen people that are paying for it. But once we have this, like up as a GA candidate and we have some of like, say the tens of thousands of developers in the world that have opinions about metrics generation. Yeah, I won't be surprised if there's another wave of like question and answer and like demand to change things. But I've gotten a lot of feedback from people that the ones who are sort of really up to speed on this stuff that like what we've done. And there's going to be a large large number of people who are only familiar with the past like Prometheus model or stats be model who come in and say, whoa, I don't get it. And we need to write content for that that user for sure. But I think it'll probably be useful to have a little bit more work on, you know, how do I make a view? Please, I need to make a view. It's important. So, so I think a little bit more is needed. I should probably be involved in view and reviewing latency changes. I do speak Python. I would be happy to be reviewing those. I had, I was tuned out, I guess, or please CC me or ask for reviews. If you're, if you feel like I could benefit you. It'd be nice for me to know at least more about what's concurrent activities. Cool. Thanks, Chris. I think I understand the request. And I'll be happy to be involved in all of any of those conversations as they're happening as well. And Gitter, Gitter is a good place to ask questions. All right, everybody going once, going twice. It's been lovely. Hope to have something to share by the end of the week on scope. It has a lot to do with metrics. And that issue 381 about binding tracing and metrics together more tightly. I'll see you next time. Thanks, everybody. Have a great week weekend. And I'll see you next time. Bye.