 Good morning recording hi Kevin Tell me if I've got too much background noise. I've got construction project next door It seems okay at the moment. Okay. Good Just working on presenting here And we'll start with PRs. I think You good all right, so goals you just go through all the PRs and issues and discuss them close them if we can Yeah, that sounds good All right Just kicking it right off. I suppose I could keep a document of notes. I can't remember if we've done this in the past search I Did this with Ted before and we didn't I don't think we made any okay I don't think we did either. Yeah, I've done this with Ted as well. So okay cool We'll just continue then so that's a good idea starting with pull requests. Well, we'll just record things and the issues This first one providing guidelines on low card Nellie spans is so new that I don't think we should shoot out of it I've approved it Make sure you yeah, okay So a number of these are kind of morning Matt, but these are Working are pretty minor Yeah, I think This is not controversial this one about the user agent It's also very new the goal here is really to work through old issues. Maybe we should work in reverse Okay Yeah, like that one. We just looked at if it's got enough approvals we should just go ahead and merge that because It's already have that Required Okay, all right, so we can't do that All right, I know there's four in a row that you are author of maybe Right now, okay. All right. Yeah, the other namespace named meter namespace is also very new and it has some approvals Keep it in order here. So let's see. We've got three approvals I think we can get one more on that pretty soon. I'll advertise this one after the meeting here Okay, let's go through yours next Okay So VM image semantic conventions Has at least one approval. Have I not approved? I have not approved yet. I can't remember. I thought I did Oh, I asked a question about host image. Yeah, I changed it. Oh I see so It had been tagged and now it's been version. Okay. Well, then I approve It looks like we still need a couple of approvers here, but it's not it's not stale at this point. So We'll after this meeting I will post a Rally to get some of these things that are near nearly approved done Common event attribute names. I Feel like there's been some lingering confusion. I know there's also an OTP. I'm waiting for my network here There's also an OTP open And there's been some shift in the PR over time Yeah, so having to do with events versus messages events kind of broadly Concluding info and error messages versus kind of like message announcements of sort of streaming or miss passing variety Do you have any looks like current state of your thinking on this? I think whereas there still needs to be some clarification on this You know with this is to me, I thought but I thought this was clear with everybody that that a Message is just one kind of event that you would attach to a span So I think that that is actually not too controversial I think what what worries people is a sort of slippery slope involving error messages and exceptions and stack traces Yeah, I'm not saying it should worry them. I just think there's we're struggling right now to get a spec released and the project off of its feet and that seems like a Like a swamp to me, right? Yes one. I'm very interested in But it's I will also interested in avoiding swamps. I know Matt who's on the call now Yesterday said something about how important that seems for a 1.0. Do anyone else like to defend or speak about this? I Would like to chime in on that kind of on a PR I think where this stuff was being talked about and I think Kevin has been leading some sort of charge and Trying to define some semantic conventions around events that represent errors I Think a lot of that lives in an OTAP, but Yeah, I think it's essential for a 1.0 So I did kind of want to check in to see if if this is something that you're That you're passionate about and would like to lead the charge I am leading the charge on that there is an OTAP for the air stuff and I'm thinking we need the OTAP Yeah, I think I think we need specific structures to hold that that you can't Really model that with simple attribute pairs and So I've actually got a project out there POC to to demonstrate That all this would work Yeah, yes, and I haven't updated yet. I've got I've only done it in Java so far So I want to do some more more languages Actually, I will briefly add that before I left Google. I was working on their errors monitoring system event error monitoring system for the Google GCP platform and so I remember going through the exercise of like writing a parser and tests for six languages Stack trace parser And then doing some sort of semantic work on them as far as sort of identifying frames and so on which is a lot of work That's sort of why I know it's a swamp. I totally support Getting kind of finer grain information about errors as a source for trace Sort of information, but yeah, I was yeah, I pulled the source code from AWS SDKs as a starting point So I don't know how to proceed on it like I'm supportive, but I worry that it it's just Like too many issues confuse people. I don't think it blocks. Well, I don't think it blocks the 1.0 because It's it's really not Changing the API I agree. It's just it's just another kind of event. So you're just publishing an event The only thing that it's going to change in the protocol itself is is now you've got to support complex objects Rather than just simple values Right. Yeah, from my point of this being a something we need from 1.0 is Right now most the six are going through this exercise of converting over data dog instrumentation and there's you know numerous points in the instrumentation where you rescue exceptions and you know are aware of them and would like to be able to annotate stands with this data. So I Definitely think we need something. I know it can be a swamp like I would settle for just You know Semantic conventions for events on a span and attributes even though that might not be the best fit for 1.0 If that could be somehow a launching pad to something bigger, but I'm also if if If having specific type events for these is is reasonable I'm I would also be interested in that but yeah, if you look at my my test project I got it, you know, I got a the data types defined and then I'm converting in four different back-end systems the model for four different back-end systems and There's there's maybe some more I'm not a hundred percent satisfied with it yet There needs to be some more detail to it, but it seems you know, I just want to prove that it works and then You know as far as the actual stack trace conversion I need to do it in some more languages to make sure my recollection involves C++ a lot because it was a very important language at Google and It was hard to work with C++ stack traces because for performance reasons you were inclined to just record program counters and then it requires a symbol table and and a shared library address mapping Which is a pain just because across your whole system You have to keep remembering that you have to find a source for these things that you can't symbolize them without that source and so on so I would be inclined at the very least to To completely separate the issue of message events from errors and stack traces and it looks to me like right and it is On this PR. It's just PR 397 is originally mentioned errors and now it doesn't so so Then that doesn't mean that people are completely satisfied with it yet. I am but Right because the I Go ahead Yeah, the only comfortable controversial thing or thing that's not not understood is this message ID, which comes from gRPC This is where that comes from because on a gRCP, you know by bidirectional Oh, right. I you've got messages going back and forth, but it would also apply to something like like HTTP server sent events possibly Are the do you know if SSC HTTP server set events are Are sequenced like with ascending numbers the way you describe here? I Haven't had a chance. I was gonna look that up, but I haven't had a chance to do that yet Because I would be inclined to just loosen the semantic convention to say they're identifiers They're not they're not incremented. They might be depending on your system. Yeah, that's a that's a good point Yeah, so I will do some more research on that And maybe depending on that research would making this required I Guess you think it should still be required might be something to Investigate during the research should be optional right Okay, so let's move on From that I think we should try and get this the messages stuff separate I think that it is now the old that still talks about Stack traces. So if we update the O-TEP to not discuss some messages then as well, I think it would be Like another round of update on that O-TEP might help as long thinking Yeah And to close on that that error discussion like What do you think about us at least kind of trying to get some airtime for that at the spec Meeting just kind of bring it to people's attention again and let people know that this is Still out there. This is still kind of a thing. We're interested in solving so it sounds good to me I'll post these notes and we can remember to do that or or maybe I'll make an action item at the end of the Daying for someone to put this data into actually I know what I'll do We'll put this link into the agenda Maybe someone can multitask that like by the way by the way can we can I get the link for this document? So yes, I was just about to put that in the chat. I can never figure out how to use zoom once I'm sharing my screen Maybe one of you can I'm gonna slack Carlos outside of this out of ban here Carlos, can you put that link in slack somehow or in this chat? Sure. Okay, so Thank you Matt. I agree. We should we should discuss that Okay general identity attributes This seems like it's stalled out, but I don't remember any controversy. I mean, it's a little bit slow right now getting there Kevin do you have any thoughts on this one? I Think this needs to get merged. I don't think it's got approvals. Yeah, I don't think there's Oh geez Approvals, can I give you one? No, I've already approved it. We need one more approval on this We can approve that one. Great. All right, let's get this one you this one done. I don't recall any remaining controversy and user This is cool. I appreciate this in here I Mean at the very least we should just be merging things so that they're in the spec And then people can raise questions when they're actually in the spec not buried in a PR So we'll get that one merged Oh by the way, Josh, by the way, Josh, you need to actually You know, I cannot actually see the document with you of course Sorry I don't make that anyone with a link can edit As of now, I believe I've done that sharing exercise, sorry, right that's Okay We just looked at asynchronous messaging span attributes This I just got a message from Armin this morning says that Dynatrace is working Internally on their own list here. And so let me get with him. Okay, probably This is a list of merged one. Yes. Okay, okay Cool Okay I feel like let's see And then our PC cement conventions is a month old Yeah, and this one doesn't seem very controversial to me Yes, it's also quite small. So are there Okay, this is ready to merge Just needs one more approval. Well, I think that could be me No, I try to approve these things as fast as I can somebody approve this please Carlos you can do it Yeah, sure. Okay I mean if you approve that is Okay Let's see transformation to Zipkin, I seem to recall this I approved it And there's this question about server-side client-size span IDs little typo here Okay This is saying How to handle span kind how to handle starts There's some to-dos about how to handle things that don't exist in Zipkin I'm mapping from span kinds. I think this is it doesn't look like this is controversial to anybody It needs votes Okay, I'm does anybody have anything to say while I put a note Yeah, do you do what the to-dos and it looks like it's not done. It looks like it's not done Yeah, I hope to get this merged before adjusting other items that needs it. Do you mean the to-dos? I mean just the to-dos. Yeah, I see okay four or five to-dos in the back Well, I think that he should let's rephrase these to-do items as some kind of undefined to be determined future That's vacation the fact that we have these to-dos That would anyone Disguise with that. Thank you. I'm just worried about merging stuff in with you know Even it's just the word to do because I noticed that we did this a lot earlier this back And then we still have some places where we've never come back and clean this stuff up. Okay. I Agree, I want to get to the issues, but let's continue with the PRs My network is too slow, I'm sorry There we are flush interface. Yeah, we're still talking about things that need to be talked about I guess So this was discussed at length Ted has some concerns I Do too, I don't think that should be an API I mean their concern was us if you run serverless that you need to get it flushed out Yeah, you need a specific call to flush at the at the end of your Execution so that it gets pushed out because on a lot of platforms that instance sticks around even though your execution is fixed but you know the other way to do that would be to put some kind of a venting thing in the SDK where you could listen to Yeah, the execution So yeah, whatever's coming from from the from the platform and do it Yeah, I guess my attitude is that this Interface is going to be now I Don't have us. I don't know. I I understand both sides of this argument Making a flush API be a specific implementations exposed functionality seems fine making it part of the broad default SDK interface Does seem to worry people for legitimate reasons I Added a question about on shutdown like many some languages have a shutdown hook and some I think of the cloud providers We're doing serverless provide hooks as well. So Okay, it's not clear to me that this is a great win or very important Name else like to defend it in my ways. It's okay to say that we aren't going to merge these eventually so Yeah try and Get some update on that Um New Spanish should always be created for service bands. I feel like we resolved this one. So what's the hold up? Okay, a lot of people are saying we don't need this. Well, I'm not sure what you say Yes, that's what I was trying that's what I was realizing this pure states that we support the W3C So thank you Matt It started out as a zip-con specific issue, but okay, I personally am not inclined to go through the last four of these Tristan we're here. I would I would be happy to talk about 354 I am personally responsible for 347 and I'm trying to catch up with it, but it's not it hasn't moved Other than to discuss some terminology renames Which we've kind of agreed upon in the metric sign meeting, but haven't haven't applied to this document yet Chris do you want to talk about probability and sampling I I Am there's two here on sampling once quite old I Personally have feelings about sampling, but I've been staying away from them in this open Tometry context Yeah, I think I I mean this is pretty old. I was waiting for a decision from the W3C repo I just need to update this like basically the change is that we're taking the other half of the The choice idea to make the sampling decision Okay, I think it's so are we still waiting? No, I just need to go and update it. Okay This was whether to check the high bits or the low bits. Yeah, although I would love if that were the only problem I think what's gonna happen is I'm gonna update this and then I'm gonna get the same feedback that Dispection prescribed the sampling algorithm at all Which There's sort of like a an SDK spec and an API spec and I think the API spec should probably stay out of that Language, but the SDK spec is sort of saying what we're gonna do in the real world, right? yeah, although I think we have this problem with the API and SDK spec and It's like everything that we don't think is actually part of the spec just gets lumped into the SDK Whereas actually the SDK has its own API, but it's like the part of the API that's facing you Yeah, no, I understand I agree too But we're not very clear about saying it's an API, but it's an SDK API I will come back and update this to you I like Yeah, I'd love to disclose it now, but I think like either I will just like write this up and Justify putting it in the API or move it over to the SDK like as it's written but changing the left bits to the right bits Um, okay, and then this last one I'm sympathetic to but I think we just need I'm As far as priorities go it's not priority for me. So It needs someone else to Shuffle it. I think I think you know, it's a Bogdan said that we are not gonna do this So I think it's okay probably Okay, that was it for the issues or sorry for PRs How did we do this in the past sir by least recently updated Hopefully we can get Through some of this stuff Everybody ready? Okay TLS support for exporters Quite ancient TLS what's the stand for? Okay, so this has been mentioned as being related to where it's at spans Does anybody have a feeling other than yes, we should have TLS support That yeah, it seems like a given but I don't know Why this just seems like a sticky subject that needs to be in the specification Yeah, I Guess if It kind of depends on you know who you're exporting to But I don't know why anybody would not support that It's I Think the question has to do with UDP transport which is common for oh, yeah, and Jager So I'm gonna see we gotta get rid of Too many issues Okay, open telemetry to Jager span transformation. This is oh, I want sort by Fender API compatibility, I can't imagine what that means Okay I'm gonna say Tigran show this what should I say I Was 115 merged Yes, okay, then I mentioned SDK API I felt like we have these recommendations Library guidelines Anybody disagree We'll see what he says I Consider specifying the maximum value of entry TTL. Okay, so this has to do with context propagation It's an open topic for getting the spec written on OTP 66 So it occurs to me that we should have some sort of label for things that have to get closed and the beat 0.3 milestone I'm gonna pretend I know how to handle that Zero milestone Label It is a milestone and It's already milestone good Okay, this If you I don't know these milestones apparently Okay Good that we have that milestone Jack Consistent tag attribute resource namespacing and structure. So this was my issue. I'm gonna close it. Ha We have added That's where we have decided to use the names Tracer and meter as a name space we have Decided not to include resources in the one point out of release as far as I know this can be closed But that's progress I The spec for handling baggage. This has got to be on the milestone as well. See if it's got that milestone it does so Reading this let's see See Okay, cool, I do I disagree Sampling decision may need update trace state normally attributes This is another one of these cases where I'm feeling a bit of length because I Don't see how you're gonna use sampling decisions without sampling weights and we have a brief punted on weights. So Like it's not really sampling if you don't have weights my personal opinion So I could be wrong about this but I think this this came out of the Otec the Bargain just revised to remove this Which one which Otec was that But this is this is all came out of like Yeah, okay, okay, I kind of agree with that Yes, I might my feelings that I could be wrong about this But I think by removing sampling hint We're actually like changing the the order of the sampling decision So before we'd make the sampling decision include some information in the form of the hint and now it seems like we Don't need to like pass extra information from the sampler along with the trace state If that makes sense, so we set the sample like decisions separately Okay, I like that I need I don't want to read this in the very moment, but we'll we'll link this Yeah, I think what I'm gonna do is close this with a reference to And it's it's this one. It's up 79. Yeah Okay Okay, I'm ruthlessly closing enough issues. It's really easy to reopen them Okay Limit on number of attributes links events behavior. So this is part of the SDK specification I Have lots of feelings, but don't really want to fill the meeting with them. I Haven't wondering if these actually belong to the milestone, you know If I think we can remove this from the milestone is what you're saying. Okay And I don't know if there's anything controversial about it. Yeah, I think it's They're setting for this in most of the SDKs now, I think I just could call us not a blocker Indicating issues on spans and metrics pretty old we may need to have a language that's just had adding the information attribute All right Anybody have a feeling I'm about to say that it's not in the milestone Same. Yeah, she's really Yeah, I need to be in a milestone. This needs more clarity Patient yeah, I think we definitely need to do something here about Proposal I kind of want to close it, but I'm not gonna be that maybe it needs a needs Does it have me discussion? I Yeah, I don't know what else to say Discuss using htp request and htp response Um, this looks pretty old Um, I don't think this list that feels stale enough that we can get there's close it. Yeah Kevin you're more aware of the current state of specification for yeah, there's not there's Right, there's not any semantic conventions for headers anyway right now Okay, so yeah So you would have to add that first, but like this says it's spent kind Well, no, that's not true because the span kind is whether it's coming from the client or the or the I mean That's this seems pretty more the server Yeah, I think this this is old enough that like when possibly when Daniel opened this ticket We didn't have semantic conventions at all. Yeah, probably it's more relevant now than I was when I opened it That's that's true. Okay Or else submit a pull request Ruthless ruthless Version labeling schema for API and SDK. I remember this was discussed I thought we had just I thought we had resolved it See the proposal here Well that proposal got merged. Yeah, okay, then I think we can close the issue. Can't we resource resource All right, let's see next is one Carlos you you found this It's this is when like whether we should actually allow users to have a single method for adding multiple attributes I think this is not a blocker and then I think it's still and I need to Feel like we actually have discussed this in other issues in PRs the idea that we Have added support for list-valued Attributes at least in the call Yeah, well, it's not about adding multiple values to a single attribute but adding multiple attributes at once And it's more like sugar. Oh Yeah, you know what in the go API. This has been done already. So Yeah, let's I mean, it's yeah, I don't think it's important. I really would like to grab this up But yeah, it's not let's not make it part of this milestone In my opinion, this is not needed. I'm gonna close it. I'll let you reopen it if you feel like it I haven't missed on you a ruthless ruthless We're back to sampling hint And Now we've decided to Undo that so I'm gonna go here and say closing this According to the Otech here Ruthless, no fun anybody else Is spam context the same during the entire spam lifetime? Oh This this sounds like a swamp Some implementations need to change the sampling flag during lifetime of span No, it's like you can actually replace it. Oh, I don't know actually Oh, yeah, I I could see being able to replace Spam contact on span context on a span, but the span context should should be immutable Actually, I don't got around this by changing the way that we created the spans And so we make the sampling decision when we create the span and so hopefully you don't need to change the sampling flag after it's created Yeah, I I Think maybe that's the reason why the sampling API takes Make the decision before making the span Write something here closing it and so you can always reopen We've closed about seven or eight issues already Is customer random generation requirement for yes, okay or not this is an argument again for putting the the sampling decision Spec changes in the SDK instead of the API Do we want like if you are x-ray for example, you don't if we put this in API your choices either violate the API or like Amplify part of the SDK so so would you say we we believe this is not a requirement because we've removed sampling hits from the API Yeah, and why we did it in Java and I've already got I already put in a There's that there's a there's a Generator or generate IDs interface and then you just swap in different So we have an implementation to do x-ray IDs So a big part of the argument to put it in the API is that if everybody's using the Like like ID generation Algorithm then we get the same sampling decision on the way across the request So I mean we can talk about this in the ticket, but I think we're still gonna have this So it bleeds over into that question about sampling From W3C is that correct, but if you're using if you're using x-ray you're probably Using the sampling decision coming from like the AWS API gateway Most of the time. Yeah, I did think if you're using it. What it means is that we like Wherever you you switch from services that expect x-ray to services that expect, you know your your own sampling algorithm. Yeah, yeah Yeah, that's true Yeah, I feel like sampling is something that should not work in a coordinated way across vendors. It's too complicated So that doesn't bother me what just was said I Look, I'm more familiar with people wanting to generate their own random numbers Random numbers just for performance reasons like you have a thread local generator that you can access without a new text That's the one you're gonna use to allocate your spin ID so that you don't have contention just getting an access Just getting access to a number generator But it's not a requirement It's just an optimization that you could apply if you decide that you want But I I would I believe we could still we could close this and still require that That the standard SDK is used the high 2064 bits for example of this the trace ID or whatever the other issue about W3C says Yeah Plus we have some freedom here because we can also you know, we ship our own version of this with the SDK so Yeah, the SDK already has this control I just don't see why it would be a requirement unless it's related to sampling which would case We can put it in that other issue Canonical type field for spans. Wow. This is a very ancient one Do what where do we land on? Component I think this is one we ought to resolve in this milestone This needs discussion I believe and I'm gonna put that back in the notes I feel like that's a swamp Probably one worth we're looking into Yeah, but we need to resolve it because I know when I wrote the the extra AWS X-ray exporter It's got the the X-ray models got different sections for different kinds of spans. So You may need something to switch on You know, I feel I see this attribute. So You know at this section like one one thing that I've had experience with and trying to create like a taxonomy of stands is that you have some gray areas like one of them that often comes up is you'll have like Database clients that operate over Kind of like partially like this HTTP REST API, but they're also kind of a database and like yeah That's a good point. I know kind of like the The attributes that apply to both HTTP and database are kind of applicable there But maybe that's a new a new thing and a tax on HTTP Personally feel like the right way to do this is to have schemas and a thing can have multiple can satisfy multiple schemas Therefore, it's multiple types of thing so you'd have a schema saying to be a error message you must have a stack trace and And a type like I don't know like you just give a description of what types if these fields are present Then you are a this type of thing and then you can make inferences Yeah, that that would work because every one of semantic conventions has some required Stuff for that So you would you'd have to look at multiple things rather than If you did it that way you'd have to look at multiple Attributes so the type can be union Yeah What should we do which if it's a union you just need a hand figure out I mean you'd have specific handling for each situation more than likely Yes, you can define a type as a union type So it's it's I this feels like a very great area that I don't care about enough to get hung up on so I would be in favor of Striking this myself, but I don't want to speak here. I'm not being with this right now. I'm not sure why So we'll leave it in the milestone and move on I Sink and async children follows from I feel like this is stale. I feel like links solve this possibly Yes, log correlation attribute names and values What's we've got log logging exporter I See this as a request for semantic conventions that we can use externally to Open-tometry, especially for span ID or span contacts Yeah, that's a good idea, you can you can sign this one of me if you want to get it in for this last time Sure What you asked for do not recommend URL path as acceptable spending so Yuri has opened a PR about this. Maybe he's yeah Okay Yeah, yeah Okay Remove has remote parent and spit from sampler. Show sample method. I Have I somewhere? I don't remember where I recommended but in no context of Otec 66 You have this current current span sampling API Which has like six arguments in them and all of them come from the context So you could just have a sample with a context argument And I know there's a performance argument to say well You're an SDK and you're in the middle of this delicate operation to start a new span You've got six fields lying around you should just call the sample with your six fields But you could also simplify the API and call it with one context and now in a context of Otec 66 We have a upstream context We have a current span and you have any number of attributes that are in the distributed context or the correlation context So you should be able to use all of those and I don't feel like I but I also don't feel like I care enough I keep saying that about sampling Which is really contradictory, but So look at that and see if we can clean it up, but maybe not for v03 and Just looking at the date on this. This looks like a predates the Let me say closing this Now as there are a number of in-flight and unaccepted issues, there's about sampling This is still Nobody's gonna argue with me. I think okay describe describe Um, I Kind of want to close these two about metric exporter as being covered in 347 This is covered in the right number. Yes And I know that that is not accepted yet, but I'm being ruthless again Now we're down 12 issues at this point Document Tracer SDK. We have a document, right? Yeah This is feeling really good. We got through a whole page almost zipkin export mapping. So now there's an open PR about this Yes, this will close with 380 Feature request I think this is covered by Issue 382 I really don't remember these numbers this year I'm gonna make sure I I'm not lying in a second 197 requests We're almost out of time and I actually have to get off the call Like right now. So I'm gonna say we we did it. We did a whole page This was just some I'm skimming now This feels like it's care. There's a current PR about this, but we might not even need it You close it We can keep doing these Some of them are tricky. These two are tricky. I think we can close one of them Unspecified values is still a topic that keeps coming up. So I'm gonna leave that one for sure I'm working on the metrics. It's API specs this week So I'll get this one done. Some of these are for me. I Can do this one too So we're down all the way to Envar for where to send spas which came up earlier. So the context of TLS Oh geez resources a span attribute. Okay. This is gotta end. All right. Well done everybody. Thank you I feel like we should do this again soon. Yes Maybe not until next week Yeah, actually you like to go through the actual milestone, you know Okay, did anyone here object to Well, I I kind of don't want to do this again tomorrow morning because we've got two other stig meetings Well, at least I do tomorrow And I'm afraid that Friday people won't come but I'd be willing to come on Friday morning If that is acceptable for people acceptable for people Okay, okay, what is that javasig Friday? Okay All right, the action item here is we will schedule one for Friday morning that doesn't overlap I think that means 8 30. I might only be able to make it for half an hour at that point But I'm happy to meet you all then. Yeah, that this was great to me. Okay. I'll put that in my notes And I'll share we'll share this document properly afterwards. Yeah It's Okay, we will put that link to that on the calendar Carlos you'll edit the calendar when we get that set um, yep And I can now sit on get her. Thank you all Yeah, all right, everyone