 Hey Tommy, hey David David you there Yeah, I'm just working on my audio. Thank you. Okay. How's it going? Good morning Morning, Eric. Good morning And hey Vlad. Oh Vlad's gone scared him away. Hey Clemens Hello Welcome back Vlad Glad you there About the hoe are you there? Yeah, he done. Hey, hello Sorry, I'm fighting my laptop That's okay. Glad you can join us again. We missed you there for a while. I Ended my sabbatical That's nice. Getting back up to speed Cool. Hey, good morning. Hey Lou. How's it going? Let's get people just a couple of minutes Eric, can you refresh my memory? You were pushing back on something being required For this for the manager was it source or type or both? Particularly remember I think it was probably Source type No, it's okay. I just wanted to make sure I was remembering which one it was. Okay, well, we'll get back to there All right, somebody went flying by somebody just left All right, it's three after why don't we go ahead and get started? Before we do that though, we've been kind of just walking through this sample scenario with github And I want people understand that this is just one that I threw together if you guys have other samples or Walkthrough exercises that you want to walk through, you know, please speak up or feel free to add it to the doc or Create your own doc or something. I don't want this to be all about the thing I find interesting, right? There are obviously plenty of other scenarios out there, so we need more to walk through to make sure we're covering all the cases Okay, let's let's quickly though recap where I think we left off after last week's call And please speak up if you think I get this wrong or got this wrong We decided that we're going to add a source property and It's going to be a single value and it's going to match whatever The source that the cloud event source value is going to be so it's kind of like a filter in a sense in the sense that All cloud events must have that source value. It's going to be required for managers to support optional for clients to send it in on their subscribe request and Whether it this this required from managers or not is we're going to start with it being required and see how it goes going forward I know there's some questions about Whether it should be optional or not for managers to support it if it's absent and the request then this and the Source value and the cloud events that are sent are unconstrained meaning you can get many different values as close to just one value That sound right so far to what we agreed to last week. Okay, so really we're sorry go ahead Yes, okay, cool. Okay. Similarly, we're going to add a types Attribute it's going to be an array of CE type values meant to indicate the type of the events you're interested in so again It's kind of like a filter thing, but it's no wild cards or anything. It's just the exact values itself Gonna be required for managers support optional for clients to send if it's specified all cloud events send must have one of those types If it's absent then the type value in the C's that are sent are unconstrained And they can be anything barring any filter of mechanisms that are Okay, any questions about that All right moving forward Gonna keep the filters pretty much as we are as we have them today other than they're going to be optional for managers to support They've always been optional for clients to send but we're gonna make it optional for them for managers to support And then the mechanism by which they indicate that is still TBD This in in the discovery spec Okay, any questions about that? Okay Okay, so I've based I added this just yesterday just to make it clear or just to because I felt like we needed some extra text in The spec and maybe this belongs better in the primer. We can figure out the exact placement of it But we did actually recently out of PR that talks about this three-step processing model And I think we may need to we may need to revisit that not because it's wrong But just to make it make sure it's still aligned with what we agreed to up here because I we didn't have Source and type before so we may need to talk about in in the context of creating the event Okay, so just big verify that text We need to make it clear that the source and type are meant to be used in the first phase To make meaning during the create event phase And in particular to help identify and configure the event source or the producer with respect to the events It's generating and obviously the config field will go will play a role in that too Okay, but I want to make sure that when it comes to these two properties They're meant to be used in conjunction with config to configure or identify the source or producer Filter is strictly for obviously the second phase since it's the filter phase and then sink in protocol I meant to be for the third phase and the reason I thought this was important was because we have a quite a fair number of Attributes, let me show you an example just so you guys can see we're talking about here Right, so we have quite a few attributes here I wanted a reader of this to be able to specifically say okay, you know This set is meant for phase one this other set the phase to this other sets phase three That's where there's a very clear grouping of these Attributes and I think that will help people understand how we envision these things actually being used if they're just sort of randomly scattered It seems like let's let's clear to me anyway But then at the end of it. I'm sorry go ahead aren't the use of those contingent on the implementation of the backing systems to which subscriptions are being made You are an excellent straight manner. I guess that's why I put this paragraph right here because I want you need to make it clear that these are our design This is our design mental model. We had as we wrote the spec. However How people choose to implement it is completely up to them as long as it gives the right semantics So for example, you know, you could technically do the filter during the create phase if that's the way you chose to implement it, right? but And so we're not constraining that it's just I wanted people understand this is this is how we define the attributes In terms of how the end user kind of perceives them and it is a little bit of guidance in terms of how we kind of think Implementation might go but we are in no way requiring that the information exactly match that it just have to it It has kind of appeared that way to the end user in a sense. Does that make any sense, Eric? Does that cover what you're worried about? It makes sense. I think I think there's a problem there, but No, go ahead. Keep keep keep going. I obviously missed it No, I just I mean, it's what I said already that Based off of The system that's being subscribed to in this particular implementation of that system There's there are attributes that could be Part of one phase whereas in a different system will be part of a different phase and that's not very specific, but imagine I don't know two scenarios in you've got two Kafka topics that you're subscribing to and so in one of the Kafka topics the the events that flow through that topic are very specific and so you don't need to do any filtering and just identifying that source and the Vents in it that you're interested in that's that's a static declaration of those, you know, where things are going to come from but in the pure topic the those events are sent and then also the events that Some other kind of events are sent so there's both the static element of hooking up and then the The specific element of filtering for just the event that you're interested in So that's a case for you know, same technology everything else. It's it's user configuration specific whether that the what kind of type is going to be a A dynamic thing or a static thing and so I Don't know it seems like there's so much I It's not bad. It doesn't seem like it's a bad idea to talk about the different classes of The phases of application the kind of static ability to analyze something But it seems like all all subscription APIs are going to have to pay attention to what is the meaning of this and In the particular context where it's being applied And and do some analysis even even the same argument could have both static and And dynamic parts Can you elaborate on what you think needs to be changed in here to satisfy your concern That's a good question. I wasn't trying to propose anything. It's far too early in the morning for me minutes ago but Anyway, the I the this proposal of Stages, I I think that it's useful to talk about that being an important part of what the systems have to do. I don't know that Declaring What attributes are a fit into what? Stages is something that you can do with With a reasonable confidence. Oh, okay. Okay The way you phrase that they're clicked a little bit better with me. So thank you. So so let me ask you Let me hook on that a little You're because I think you're basically kind of implying that something like trying to say source only applies or typically only applies to the first phase Might not be accurate And I'm gonna understand is that because you think well applying the source Property may actually be implemented more as a filter kind of a thing or Do you think trying to explain our rationale behind? Why sources pulled out from the filter is a bad thing? I'm trying to just trying to figure out where to go with this guy I think I think you're raising an interesting point and I don't want to miss it. That's why I'm trying to fully understand it Okay, I'll do my best at this. So the the source Is could be itself a mix of dynamic and static pieces, right? So the source could be you know, some you know Whole string of things it kind of says here's the actual physical component this data is going to come off of but then it could also include maybe the A Particular topic in it. That seems like a reasonable place, right? but that that topic could be something in based on the implementation's dynamic and That that maybe there's multiple Topics or maybe it's a you know a client to a specific kind of I know product or something Retails poisoned my brain but So, you know, there's I want I want all the events that said coming from you know the Gucci stream and so that that source could be both the stream that contains this whole slew of Transactions and events related to products for a company and then the Gucci part of that would be a filter over that stream and so that's I don't know. This is I'm pretty kind of strapped. I feel like I'm stretching in this particular case, but It seems like the sort of thing Okay, we'll tell you what when we do this So we have a so close and I we're currently working in a parallel product integration stream together and We want to bring some of that context here and we actually have a scenario like this where Or might have a scenario like this. We were in no means by no means settled on this but effectively There's an SAP system that wants to go and deliver something into Azure and to do this We need to do effectively a bulk subscription on The the the customers SAP scope and that's something like this where Yeah, there's a so it's not every ball and let's say Gucci is a is an SAP customer, which is not unlikely So So you basically you have a subscription manager, which now which is now acting somewhere over in the SAP system And you say everything that's being raised in on behalf of that customer send that here That's a that's a legit. That's a legit case. I think and the way how we would prefer to model this at this point Certainly the Microsoft side of this of the discussion is to say Gucci's scope inside of that SAP system that is a Effective the root source if you will is the prefix for all the other Substructure that might exist in that system and if I walk up to that source, I can go and subscribe to everything that's inside of it But that would still that would still say so I would I would indicate in in in this relationship I Would go and indicate as the source the super the super scope this like you know everything that exists Everything that is this all the services that exist for Gucci in SAP And then I would not care about this type but that would be my top level selector of how I get that all those events and then at the At the second stage then I would go and potentially filter those down But for this particular case, I might not but then if I subscribe into a fact so all of that get now flows from this SAP system into a Event dispatch broker effectively and There you would subscribe on that same stream which you subscribed up from from the other system and there you would now say I only want to have events from this more specific source Which is prefixed by the the original source and maybe only this this type and this type And then I might also want to go and filter that further down to this subject prefix or subject suffix So there's the kind of this broader subscription Which then funnels events into an event broker where you can then go and Have a second level subscription to be more fine grained That's that's one example of of where source becomes effectively a scoping mechanism. Does that make sense Eric? I think so okay, okay What I was going to say before was I think Rather than trying to rattle too much on this because this isn't really normative text on the spec. It's more like Insight into our thinking process or how we expected things to be done, but it's technically not normative Eric, why don't we wait until we actually have a pr trying to address this particular task or item? And then we can wordsmith that to address your needs. Does that sound okay? Sounds great. Okay Okay, cool. So I think this is where we let let this is where we landed after last week's call Does that sound about right to people? Okay, so then what I did is I went back and revamped the sample subscribe message based upon what we agreed to um Still has the same so as the protocol protocol settings includes the secrets so it can do signing This is where it really starts to change. So based upon what we agreed to The source now becomes github.com slash cloud events slash spec and We have two types because there were two different Events that I was interested in when I want to do the subscribe and I'll talk about that in a second But I have no filters and no config because all the config are now encoded within the source Okay, now um This one it got this is where I got a little bit interesting. So when I subscribe using the normal way of browser based model for github I can give it a Issues and a push type of event Okay, now push is fine because that that's the that's a stack one-to-one mapping. However The issues is interesting because github through the web interface just has the notion of issues And you get all different types of issues right create versus delete edit that kind of stuff However, in our adapter, we actually made the type to be github.com.github.issue Dot action or maybe it's issues. I came here for sure. But anyway, the key is it's dot action now The question I had for the group here is Is our adapter wrong should we have not added the dot action? or Is this now become an invalid or not invalid? Is this now become a Can we no longer use type to do this and we have to do some sort of filtering thing if we actually do want to support I mean if the adapter is right because we have no way to do prefix matching in types No, we can't that we shouldn't Okay So the question I have is if if we don't want to do prefix matching here or any kind of wild carding Then is the adapter wrong? Is the adapter wrong? Well, the adapter is I think the adapter has all liberties that it wants because it's an adapter Like if if if there is a common github, what are you referring to with common github issues where you say Um Because it is are those events that they raise So when github raises and raises an event about an issue being created or deleted The event type Is called issues or issue one of the two. Okay the actual action itself is another field inside of the event itself Okay, and what we and what we did in the adapter is we said well, that's interesting But let's combine those two fields into one so that you can filter On the actual action on the issue as opposed to just it's a generic issue event That's why I'm wondering whether the whether the adapter is wrong and it never should have added the word action uh, I I think the adapter is right Okay If it's if the adapter is correct, then how would we recommend people model this part of the subscribe request? uh, they don't Like so so I mean adapter means means we take something that someone someone built without thinking much about um, uh How we are thinking about events here And and we're trying to adapt it and and I think we want to have discrete events for each action that might happen Okay I think from so so how if called events exist And you would design this fresh How would this look? I mean, that's what the adapter should do Okay And I don't think it needs to it needs to say here is an existing weapon api and how do you reconcile that with with, uh, um With cloud events And preserving it effectively. I mean we can do that And then the answer is yeah the adapter is It's it's wrong because the adapter should do exactly what the weapon does But if we want to go and make it Useful in the cloud event sense then we should say okay adapters adapters actually it is reinterpreting Some of the things that the weapon does now Okay, so you're saying it's okay that a native github That the native github support for subscriptions. It's okay that it's going to have types that are different than what the adapter has Is what you're saying Yeah, I mean if we were if we were because they're not they're not uh cloud events compliant at this point Right, and when they are I think the advice we would give them Is if they ask you or if they ask me then we would say yeah, it would be great if you for the issues for every particular different action or or Change type You have um, you would go and raise a different event Okay I mean this should the adapter I think should reflect the guidance that we were we would be giving One of those webhook authors for how to use how to how to Raise their events and issues seems mighty might be big of a scope Okay, uh john your hands up Good morning. Yeah You know more or less echoed the same thing right if you if you look at it sort of a Restafarian kind of perspective they could still Offer calm github issues that you have written as a fire hose As well as the more fine-grain types or whatever. Yeah types that you're calling it, right? So it doesn't It doesn't have to be a black and white Right relative to our spec Okay Okay, uh remi your hands up Yeah, I'm sorry. I had to drop last week So if I understand correctly if I want in that case all the issues like As it cannot Pre-fix. I mean I should not put any types and then I should use the filter Because that's my only way to filter on types github.issues if I want only issues event Whatever they are creation deletion or updates Okay Is it correct? Yeah, I think you're right. I think if yeah, I think if we keep this as is and people really really only want to create it actions Then yes, I think they have to use filters Uh because not for what uh, so that's because what you define is normally if it's a creation Then I will see com.github.issues.created or.create As the type because you say it's concatenation of actions and Well, so I agree with you if we were rewriting github from scratch. I agree. We would probably put created here Right, but this sample is trying to minimize the amount of changes in github itself meaning what do they do today? And natively today they only support issues They don't support a finer grained filtering through their web book mechanism. Okay, so so then your second example the com.github.push should be just a com.github.repository I'm correct because I think the event push is On the repository type, no No, I think there is a push event. I'm pretty sure push event. Okay. I'm pretty sure I'll double check Because I came up with this right look at their document, but if I'm wrong, I'll let you know But yeah, you're right if push isn't there and it's one level higher than yes, this needs to be repo or something like that. Yes Again Okay Okay, so does anybody disagree with that analysis? Okay, in that case I can buy into that Um, okay, cool. Thank you clements and all for trying again in that case Is there anything about this subscribe then that? Doesn't match other people. You know, everybody's mental model of how A subscribe would look to today's github Does it seem okay? For example to ask people to be able to construct From their from their org and repo the source URL That's you know questions like that. I think are things we need to make sure we're comfortable with Now hopefully this the discovery spec in our source template will say, you know this part and then this As a static string and then this part will be templatized. So hopefully they should be able to construct it with what's in the discovery spec But so I'm probably the lazy one but If I had to post a subscription on github in my opinion, I will not put the source because I don't care what source they For me the source is not relevant because I'm already talking to github Well, keep in mind the github webhook api today requires you to specify what repo you want to subscribe to Ah, so okay, so that box is the repo is the cloud even slash bit. Yes Okay, sorry my bad. I thought it was just like uh, I want cloud even This is the organ repo that in the previous in previous incantation of this these were under the config under organ and repo And I moved them in here Okay, any other questions comments concerns about this otherwise we're going to keep moving forward Okay, um So I I thought it would be interesting To see what would happen if somebody decided not to use types and so they left it blank and instead they tried to use filters to um to get the events that they were interested in And I think the requests I say requests because there's two of them would look like this, right? I think you still have the source point of cloud events back in both cases The only difference here is you'd have a filter that says simple and we're looking for a type matching a value of issue Actually, and technically I guess this should actually be calm github that issue So I have a Question there because I can't quite remember um because our yaml and the Excuse me for that word. Um Our white space sensitive specification and And our our our written spec actually diverge on that point on how the filters are being said So we have for dialect. We I think we landed on basic instead of simple because we decided that it's not simple Okay I can't I forgot that. Okay. Go ahead. Yes. So so that is so that is still in the so that's that's in the doc It's basic, but simple is in the in the spec But the spec differs because we have dialect and then we have um, I literally code this up today. So that's why I have a very present in my head um We have Filter come Here's the spec like never have conditions Yeah, yeah, this is this is this is not this is not what we have in the in in the formal spec in the in the yaml Well, it's interesting. You say formal spec meaning yaml. I think a formal spec has the text Yeah, okay. Well, okay. So so in the yaml in the yaml for whatever reason because I because there was some edits Let me say yaml which yaml you which yaml is referring to There's in in the same repo. There is a in parallel. There is a Is a is a yaml specification that corresponds with this as open api Oh, okay. Nobody knows. Okay. Yeah, that's probably out of date I think I think the spec is the more up-to-date thing. Let me get back to the filters here I think I think this is what we've agreed to mainly because um People wanted to be able to do ands across different dialects if you support more than one dialect Yeah, because then the question is whether Do are we then okay with because we had I think before We had filter and conditions So we filtered out dialect and filtered out conditions and then the conditions were Not further specified Yeah, I think it was an and across the it was it was ending the conditions, right? Uh, okay. Yeah, no, okay. No, I remember. Yes, it was in the conditions and Okay, so then that's out of date. Yes, because I'm wondering how for any other dialect Whether the three properties type and property and value will still be okay for that I thought we had I think That these properties are dependent on the dialect and so they'll change I thought ah, okay. That's what that says. Yeah, okay. So type property and value Okay Okay, so because I was I was I was working off the the the code generated The code generated types from that open API spec and I was Just puzzled puzzled by the discrepancies But now I remember. Okay. Thank you for for helping helping me through this. Okay, sure Okay, so going back to here I think in order to get the same semantics because it's an and in the filters You have to do two different subscriptions One asking for a type of issue And then another asking for a type of push, but basically everything else is the exact same Okay That's that's the way it would have to be today based between our spec and the way github works um Okay, and notice that You know, this is asking for the generic issue thing We just like above if we needed to filter on just create then we can we have to add a filter mechanism Okay um, of course the downside That is specifically why we have to have this list of types to uh Because that that makes it easier to to to subtract to the list of types Yep, um, yes. Yes. It basically gives you the or Exactly, right. So the question I had for the group is Maybe ask this later on but I'm gonna ask him now anyway Is this okay? Right because in this in this simple case, it's easy to say. Well dummy you should have used the type field That's what the you know, that's it gives you the or semantics But you know, what if I wanted to use prefix here instead and allow a little bit of wild carding That pushes me towards two different subscribes. So my question for the group is Should filters allow for or semantics in some way? In addition to the ant I hear you're paying clients I don't have the answer, but just you have typo in the quality financing source Oh my gosh, that that's what you focus on. Okay. I got it. Sorry I I don't think it changes my question, but okay So if you if you were if you were doing this Or if we would need this I would prefer I would prefer a I would prefer introducing like if you really need anything that is not and um I might prefer a boolean dialect Which has an or in an and which into which you can then stuff other filters That's that's what we let that's where we effectively landed with a nkp Is that we were like, okay, are we going to start introducing operators in this world? Or are we simply going to go and create containers that have particular semantics? And what we we landed on this on the model where we said we're going to go We'll make these kinds of containers. So you have effectively a filter that has that is an ore filter that Contains other filters And that yields true if any of the contained filters yield true And there's an ant filter that yields true when all of the internal filters yield true And then we also made a nut filter, of course um And then you can go and put them all into an ant And they will yield the the right thing So this would effectively be be another level of this and that is that seems that seems simpler to us to evaluate Then starting to introduce a notion of operators Interesting. So just to be clear You did you have two separate dialects one for and an ore and then another one for knots or was it all one or It's already one It's all no we have a we have a logic. We have logical. I think I call them logical filters They're called the bullier filters. I forget um Like maybe bullying is better That's for I can tell you at any moment. I just need to go and find the damn spec That's okay. I'm gonna bring up the word later. I guess the so you you you did that all in one, which is fine if we If we went down that path Would it make sense to even remove the ant semantics From this so that it's not an array anymore. It's not an array anymore. It's just a singleton or is We we made we made we made it even we we even made it Different we call those grouping filters grouping We call them grouping filter expressions To make clear that they group filters and then there's an all and any and a not So that's an and that's an ore And that's a knot. Yep Interesting and they're just they're effectively just a filter that can contain other filters And then has these evaluations semantics Now, did you guys have the notion of and sort of implicit like we have it here with the by making this an array So the uh, um, so the a to p filter is Well, um, yeah, so the a to p only allows one filter For let's say a source So we needed to have a mechanism that Effectively allows you to so so filters and this is this is what this is why this is a This this is a neat trick We have one filter to play with so we said, okay, but we want to have combinations of those so we made the filter That is actually a list And then the filter has inherent has an inherent semantic that is all and any or not And then you can stuff all other filters in there, but effectively we're what we're saying is one so in in compute terms one one One terminus can only have one filter And that filter is now expanded that the functionality of that filter is not expanded by making making that filter a thing That can have a list of filters um I'm not Okay, I got a little confused there. Well a little and that I am I'm I'm a spurt. It's on my side not yours, but What do you did you remove the array or not? Yes, so well, so we started with there is only one filter Right, and then we made Right We made kinds of filters That can be all or any or not and then contain other filters. That's how that's how we landed there Right, and so let me let me echo that back It sounds like what you're saying is if we were to match a mqp we would remove the array from here And it would just be a single thingy But in order to get the multiple even the and that we have today You would have to start out with a grouping dialect and under there use an all Right and then list out in essence this thing as well as one of the choices there Yeah, if you if you wanted to have multiple if you only want to have if you only want to have one filter there Then you just put the filter right Okay, so that's one proposal And actually let me be clear and that would be remove the array from filters Yeah The interesting thing here with the all and the any and the not is that you can obviously also go nest them You can have any of three alls Yep Where you can And then you can go and build Very interestingly complicated expressions By simply having These kinds of lists nested And this also fits with and from a from a semantic perspective Interestingly enough That also fits with And that is that is coincidence Because I didn't look at open api or jason schema So I know it's not the flow from in that direction Is that you have the notion of any and all is also in jason schema and from there has also led into open api So if you if you look at jason schema For how you define a type and what's permitted in there You have a choice of One off or all off and any off Which basically gives you like duct type options for how the type might look like so that's that it's kind of similar So it's also they're not choosing and in one of those constructs, but they have these all in any And then one option And one is exactly just all with one one inside of it Okay All right, so let's pause there for a sec What other people think about that choice? No comments I have expected opposition They're stunned How could you dare propose something like that? Okay, john likes it Okay, Lou. Okay. Thank you guys for speaking up um Okay, I like If we I do think we need some sign up or mechanism personally and I think a grouping dialect Makes a lot of sense to me and if we introduce that I then like the idea of moving the array Because I don't like treating and is special Right. Um, so I like I like the direction you guys took with all that So this has a lot of appeal to me at least initially So I like it as well Ryan, did you want to chime in here? Yeah, I'll speak up because I think this is something we talked about when we like took a first stab at this last year Yeah, this always felt weird to me. It's just the implicit semantics of the array. So um, I'm I'm plus one All right, cool. Thank you for speaking up. Anybody else want to chime in? All right in that case not hearing any objection Let's work with that as a decision and see how that looks. So let me just go up here What the heck did I do? Yeah I'll work on adding to the list up top of what we agreed to later. Okay. So I I need to make it bold. Okay, so let's go Okay, we'll see how that looks that would combine these two into one subscription which removes that concern I had Okay, now just for fun, I decided What would it look like if we actually use this mythical SQL query language? Oh, Manuel asking do we need one of? Do we? Okay, so just the question is And if I remember correctly Manuel, correct me if I'm wrong here that one of means Exactly one of the nested Expressions matches not so if two matches doesn't work zero match doesn't work has to be exactly one Yeah, that that would that would swing with what Jason Schema does Correct So what do people think of that? Do we need one of I have I have no reason to say yes or no I don't either so Manuel since you mentioned that were you just saying that for just completeness or because you actually think it's going to be useful Yeah, I think Clemens mentioned something that it can be substituted with a combination of something else But I didn't get it. I'm still thinking hard about this It's it's it's possibly I'm wrong Um, and and so yeah One of might be might one of might be useful Edition But it lies in a stream of events or subscription Did I say that again? Yeah, still yeah so still if I think there could be a useful example So, okay Any objection to adding one of Oh, man, John Yes, that's that that's that XOR But this this is where this is exactly where you're breaking into jail. It's like one of those it's like one of one of is one of is I can see how you need this in Schema where you are literally um, especially in something like Jason where Or where the kind of everything is duct typed and then you really need to go land on exactly one one type looking at at alternatives um For filtering Not so sure Yeah, I'm I'm leaning a little bit more towards I want to understand the use case better because I'm just filtering down to just one exactly one event type and trying to understand why you'd need that as a filter But I also I'm okay with adding it if people think it may be useful and we could always remove it later but It's more obvious It's it's not just type, right? It's not just type So it applies to different fields of a cloud event And I could want for stream of cloud events where either one of the filters I'm applying It's true Okay, you twisted my arm enough that I'll buy into that I I personally find it a little bit weird because with with Jason Schema you're you're like it matters You're selecting the schema that applies, but do we care about selecting the filter that applied and just feels a little weird And well or anybody else want to comment on that? I just simply so an example where I filter for type and subject They're both basic filters nested inside this new grouping Dialect and with the one off I could select for events that Have one subject exactly matching what I'm looking for or a type but not both at the same time Do you have an example of when you might actually not want both? I couldn't think of one right now I don't have a concise use case. Sorry okay, well I need to kind of get a sense from the group because obviously it's not a clear yes or no from everybody Do we would we prefer to put it in and see how it goes because obviously if it's in we're going to be forced to implement it Or would we prefer to wait until we have a more Concrete use case to add it in I need to get Adding stuff is easier than taking things out. So I'm I'm leaning towards making a note and then And then adding it Like I'm not opposed to add it. I'm just opposed to adding it without having clear a clear notion of what of a compelling use case of what we would need it for I agree with this. Okay. Thank you class Anybody else want to voice an opinion one way or the other? I'm on that side too. It adds complexity to added so until we have some reason to shouldn't Okay, men. Well, would you be okay with holding off until we get a better sense? Okay. Okay, so hold on Okay, cool. Thank you. All right Moving forward then just for fun. I decided to see what it would look like if we actually use the mythical sql dialect that we've been talking about And it seemed fairly straightforward. Obviously this would need to be calm That github.issue And calm.github.push, but it seemed like it was fairly straightforward to do Does this look like I'm sorry go ahead Yeah, so this Sorry to answer your question. Does this look like yes, it does look like right But I have something I have something else that I'm and and you are typing so Um, I would like to try something something out and that is Um, the Instead of formulating the filter like like you do now Um, we could and that is um Like the filter object inside of the filter object have the The filter type be the key of an of the object that then describes the Um The details So instead of I wish I could type on your keyboard It what did you you can go to this doc here here Here's a little Okay, great. Ah, yes. We have the google thing. It's a google thing Hang on where am I? Uh while clements is bringing it up. Um Lou your um I don't know the answer to that question We I think we may still need the sql dialect just because some people May want to use an sql processor that they already have in their back end system Because that makes it really really easy for them. So for example, I could imagine somebody saying, you know what? I'm not going to support this grouping dialect that Clements came up with I'm going to just use a sql query because all my events are in a database And I could just slap in this expression to my current sql engine And it just works So I could imagine when I want to get rid of it But I could imagine people choosing one or the other That's just me I think these the sql dialect can do Can do um More than than these come but these Then these uh combinations Um Also because the proper language because the state has things like like and so you can do Easier expressions if you follow kind of the and my mind is obviously with a jms like business selectors So those are more powerful But um, you can already achieve a lot with these All and any and not and then simple these these basic filters that we have So I think I think we can get a we can we can Probably start with um With having those expressions first and then add sql if we needed um, I just want to go Just do the thing that I've just had in my mind here which would then I'm just I'm just trying and just trying to make this a little bit more approachable here all It's always fun watching other people type Yes, especially if they're very bad at it I have not learned that in this entire career um Where you basically get where you basically make the filter so the filter needs to live inside of an object because otherwise the key would not be unique but It can then effectively once you have the key Then you know what's following so all is always an object and any is always an object and not is um probably also Sorry, all and any always arrays and not is also an array is it um Yes, and then um and then the individual filters would then have objects associated with them Which then have the conditions inside of them That's that's a more concise way and kind of gets gets rid of the dialect thing because we're going to have some Now now we're going to have basic And if we're basic in sequel and then whatever logic then it becomes a little Like this makes it more compact. Okay, so you're talking about just a syntactical thing right now Yeah, just just thinking aloud and um, and not necessarily wrong sequel, but I'm just using it as an example Okay That that might be that might be stomping onto people's feelings on On how Jason ought to be looking like So I'm I just want to I just want to throw that into the group and say I'm not particular about it, but that would make the Jason more compact Okay, what do people think about that option? And while you're offering you did you want to comment on this or is that an old thing? Yeah, no, um, actually I want so this still requires The sql to be embedded in an object. Otherwise you cannot list more than one sql Uh inside the the all the way Well, it's it's not even syntactically correct. It needs to be an object that then has a simple parameter. Yeah, that's what I mean Yeah ignoring typos, I think I think Clemens point is let's get rid of the the dialect thingy here and just make this the The field name or property name or the object name whatever it's called well, it's it's um This is just sugar. Yeah But I have that I have that I just want to put that up up there and say If this especially this is this then kind of more I think Jason's schema Does doesn't it do that do it like that? well we can So let me ask let me ask this while we're we're thinking about this So what do people think is this stuff in Pink or whatever color that is is this a direction people would like to consider? I'm not saying definitely go with it, but if we think it's an interesting option After today's phone call and I write up again what we've agreed to I can list this as You know another alternative to this a little more verbose syntax that people can see it in action and And play with it. Um, I guess so I guess my question is Does anybody violently object to this as a consideration not agreeing to it just telling it to consider Because I agree with you Clemens. Like I'm waiting for Scott to show up on the call. I could imagine Scott barfing all over this Um, so Jason's she actually does it like that. I just need to go and find the right. Um, I just need to find the reference Okay, anybody want to chime in in terms of whether this is an something worthy of consideration or it's just completely insane So this if I translated this into human words, it just means that all of the following must be met, correct? So long as I can speak it in human well from just looking at it I'm happy That's one way to look at it argument that I might have is um, if you wanted to if you wanted to create a json schema for the um descriptor here Would it be possible if your keys were Describing the dialect Jason schema itself uses that so I think as I just as I just just There we go So so Jason schema is using that construct and I don't think that you can describe Jason schema and Jason schema Which means but don't ask me how I do this Um, I'm just assuming that you can so that's that's what I'm that's the inspiration for what I just what I just uh said You can't do that Hmm You can uh Validate Jason schema with Jason schema. Yeah, that's that's what I was assuming I would have those those uh, those folks now are up to version seven now Oodling on this so I was I would have expected that they can And of course we have the ever popular one of yeah yes That comes from right And then whether we whether we we use the off or not then this also The question but I think I think that's a that's a very attractive Model of just combining these things and we can go and use the same thing for filters And I think the syntax is also the syntax is also okay Okay, anybody else want to chime in on whether We should consider this or not. Otherwise, we're probably going to list it as an alternative and then review it again next week Okay Thank you Clemens. I From a straight syntax perspective, it definitely is more concise And I like uh Just whether people can actually program to it because I know a lot of people like tooling and stuff And I don't know what where the tooling can handle that gracefully Yeah, that's that's the that's the only that's the only concern is that um tooling might be a little upset by this so we we'll have to go and take a look at what the um I mean The easiest the easiest is we have to be able to express that in jason schema Because otherwise we can't formulate our open api for it and that will basically show how weird that ends up looking right So I I can I can volunteer to go and um and try to define that An open api Yeah, because I have to I have to I have to futz with the open api anyways because I'm I'm literally coding to it not right now to for the for our interop effort and so therefore I and and our our our open api spec in the repos is outdated so I have to go and update that so I would go and make a make an Make a version that that uses that All right, cool. So I'm gonna I'm gonna so Let me make that concrete I'm gonna commit to update the the open api spec to what we agree here including that as an alternative notation Oh, okay. I like that. You'll commit to it. Okay, good And update Oh, you made it all caps. Well, great. Yeah, you you've created other things in the past and and I'm still waiting for those So we'll see All right, okay, let's go back. Okay. So this is actually a good stopping point Yeah, I had to have more in the dock but since it's time for a regular phone call Let's go back over to here and Kristoff. I got you. Thank you. Um, just roll call before we go back to the other things. Um Uh, let's see andreus. No, andreus dropped Okay, no, john last while you're there Yep All right, lucas frank. Uh, yep here. All right, christoff. I got ginger Yes, sir. Yep, jesse here And lance Hello, hello, and the other lucas Uh, yeah, I said yes for the other one. Oh, sorry. No, this was too. That's okay. Um, All right matt hunt you there Yeah Okay, and all right scott you there Dun dun dun Hello and teamer Here. All right. Did I miss anybody for the roll call? I think I get ready. All right Up to 23. All right. Let's get on with the regular agenda before we go back to the fun Um, all right kukani you last chance anybody want to volunteer to do with remi All right, not hearing he's going to go it alone. Cool. Thank you remi And we'll talk offline about the material and and stuff like that to make sure Um, everything's covered appropriately Timer Were you going to grab the serverless working group session? um What do you mean like as part As part of what we did last time as well Yeah, uh, well, yeah, because because we're gonna obviously we get I need to actually i'm pretty sure we get two one for cloud events and one for serverless working group Assuming we do get two you and I talked offline about making the serverless workflow take over the serverless working group session And whether it's just you or me or not. I just want to make sure that you still want to do that, right? Yeah, definitely if that's a still of stability, they'll do really nice Okay, okay, so we'll talk offline about whether it's just you or both of us or something like that. Okay, all right Anybody else have anything relative to kukani that we need to talk about or ginger. Can you think of anything that i'm forgetting? Nope, not this time. Okay. Good. Thank you All right, um this week we'll be talking interrupt discovery at the top of the next hour So one o'clock eastern I don't think we have a lot to discuss but just in case people do Timer anything you want to mention about workflow to update the group Not much. We just finished the looping structure stuff That I mentioned last week and I think we're just looking over all improvements before the next Release of the specification All right, cool Any questions? All right. Thank you. Um Moving forward a couple of prs So this one we skipped last week because I thought the person I was chatting with may want to chime back in but he didn't So I think this one's ready to go This basically just talks about oh, we never did mercedes. Okay. Um, yeah, this basically talks about um Hold on a minute What is this one about? Oh, okay. I'm sorry. I was getting confused. This is about dealing with errors and basically this is just saying In the primer that errors are just like any other cloud event And that's about as far as we go I'll let you guys read it in case you have any chance give you a second there Will I take care of something in the background here? All right, and then down here. It was just a syntactical cleanup or anything else Again, this is in the primer. So it was not normative Any questions or comments concerns about this? Text oh, I did also add I think two weeks ago this second paragraph based upon that person's question about The adapters and stuff and how they relate to all this Okay, any objection to approving? Okay, thank you all All right next technically this was open today and I'll if we like it, I'll wait until end of the week to merge it but um Last time we talked about I'm sorry during the last design session Somebody and I may have been classic I remember noticed that source template was missing In the discovery spec in the pseudo schema section So this is just an example of you know in pseudo scheme what the subscribe looks. I'm sorry what the discovery looked like Our discovery service looks like Um, so I just added it in there. It doesn't actually change anything the spec just adding to the pseudo schema I just reordered these url in the spec itself actually appears after name. So I moved it down Um, same thing here url comes after it in terms of the definition So I just wanted them to be the same and then I just removed some tabs here So really this is the only real change. That's not syntactical. I just added this to the pseudo schema Any objection with that and if everybody's okay with it I'll wait until end of the week to merge it and I'll let people know they have that much time to review it But it's just a syntactical thing Any questions concerns? All right, cool. So I'll wait until the end of the week. All right, cool. Thank you Um, hey slinky. Um, is there anything you want to talk about relative the query expression language? Uh Waiting for feedback Okay, did anybody get a chance to review it or have any comments on it? Okay, I'm going to interpret that as Not that we're not interested in it, but rather everybody's busy And that we should just defer this until people have a little more time. Is that fair? Is that a compare or match up with everybody else's? Yes Okay, I have to look at it In where math so I will Okay, so please everybody when you get a chance to review that All right Now what I wanted to do. I'm sorry. Are there other pr's? That needed dressing. I believe these four down here are still waiting for updates I did not notice any depth they go through and I can't remember for sure who owns them other than I know for a fact Clemens at least owns one lance owns one and then Clemens you have a to-do on one that you don't own But you promise some some updates or proposal related to it. I can't remember who else is involved I know at least you two are involved. So you're being nagged. Sorry Okay nag taken nag taken excellent. Okay Anything Before I jump into these issues, I just want to see if we can get out of our backlog Any other PRs or issues people want to bring up? Okay, let's do this I don't see jem on the call. Okay. So somebody noticed that the There's a problem with the protospack in particular with the relative to the vanity go package stuff I know nothing about this stuff Netify Or the protobuf stuff Is there somebody who can take an action to actually create a PR to address this? I know a bit about this Do we want to okay? Do we want to take the action to do it? Sure. Yeah, assign me and cool. I sorry. I never saw that my GitHub events are a trash fire That's fine Thank you very much. I appreciate that Can you actually do the assigning? At the top. Let's and then that then it shows up a little more prominently for me. Oh, there we go Yes, thanks. Cool All right. Thank you scott appreciate that. Um, all right This one So this person is complaining saying jeez. It's annoying that we don't allow Dashes, and I think he actually talks about underscores too. Yeah in our attribute names And he would he wants to know if we can extend it I'll let you guys actually read this So I know I know Was that clemens? So I know I know when we talked about this ages ago We came up with a very restrictive list because we looked at a whole bunch of different protocols out there and Tried to find a a subset of the characters that would work, you know, as broadly as possible Um, I honestly cannot remember for sure whether dashes and underscores were problematic for particular Uh protocols or not or whether we just said let's not even risk it and just not go there Um How do people want to respond to this? Do we do we want to come back and say sorry? We discussed this we thought about it again, but nope. We still made the right decision Or do we want to say maybe we were too restrictive? uh Go ahead No, I'm saying the former the one you said, I mean that's uh, I don't see any problem We're not supporting dash and honestly the the v1 sdk there was a bug So you're okay keeping it as is Yeah right, okay I'm fairly sure that we have Uh, that we discussed this in issues long and extensive Um, so sure we did do Yeah, let me I'll let me go and find so keep going and I'll I'll I'll try to dig it up because I'm I'm fairly sure that we that we documented this somewhere either In issues or in pull requests why we didn't support that there was a lengthy debate about this um Okay Okay, well clements is trying to do that anybody else want to chime in I seem to remember that we didn't sub wanted to support it after we made the change in zero three because we didn't want to Uh conflict with the zero one and zero two specs What do you mean by conflict I I don't remember that well because the the dashes turned into the substructures inside of the extensions They're basically like pathing elements And so to remember something like this two years yeah, so then so when we Removed the ability to make extensions that created sub directories in the jason structure We uh limited the character set to just you know be flat extensions But maybe it's okay We could loosen it now because we don't necessarily support the zero one and zero two specs So you mean because we got rid of bags dashes are no longer a problem That are guessing I should say that's my thinking yeah interesting. Yeah, there is a there's a So dashes are a problem because um dashes are not allowed in language mappings So they cause problems so they turn into something else Sometimes the underscore character, which is different Um And then they become confusing because then you know, what are you going to do with the underscore character? Does it say an underscore character or does it become a dash? and then underscores not allowed in All the constructs like I think you can't use an underscore for One of the header types And what's that a mqp or a or hgp somewhere the underscore is not allowed Like it gets it gets ah There it is You can thank john for that. He got the link. Yeah, that's the missing thing And then we had it there I found another two bug 265 So I can blame christoph Mm-hmm So so we had so we had a bunch of back and forth There's and then there's a warning there's 287 that that's being linked taking the motivations for keeping sentient attributes minimal so scott since you mentioned the The nesting stuff um Do you have any preference on which way we go with this? I mean Now that we don't have the nesting. Do you think we should consider adding dash back in? Or do you still think it's probably safer to keep it out, especially given the other stuff? Well, well if it can't map to the other protocols, I I'm a little nervous about it because it's yet another way that It's not actually lossless Transporting stuff between different protocols right Okay, so let's Do this. Does anybody on the call believe that we should consider Opening this back up. Okay. In that case Um Clemens, do you want to try to answer this this person or would you like me to? Um, I think you might a little more what? Well using to have a little more background in terms of which protocols it might be problematic for Uh, yeah, let me I'm gonna open this and I'm gonna I'm gonna do this. I'm gonna do this probably Okay, and and I'm gonna do some research on where I think I think that was a problem Okay, so I'll say you'll respond but yes, we're not going to reopen the issue at this time I will close as by design with comment Yes All right, any objection to that decision on this issue? All right, cool Thank you all and now here's another one to have a blast from the past Um, so this person what is the name Francisco? Wants to add Wait a minute. Can we get this one mixed up? Oh, yeah, okay. Yeah, so this person initially said, hey, why don't we add a correlation ID basically? And I tried to explain that we've been down this path before we couldn't come up with a single definition that satisfied At least a majority of the use cases um And since we couldn't get agreement on what a single definition should be we decided to punt on it and people couldn't always add extensions for it however He does specifically call up microsoft assure And he does talk about it being you know picking out their definition is saying oh, they're all related to the same uber operation And just last night I responded saying well, that's great But that's just one definition because other people look at it as potentially a parent-child relationship Not just part not just related to the same uber thing. And that's exactly why we decided to punt on this um So I'm inclined to say we should keep this one closed as well, but I'd like to hear what other people think so slinky your hands up I hope I'm gonna say I'm not gonna say something stupid but Correlation ID to me sounds I mean um We have the partition key partition key is somehow I mean the the key of the message in systems that have partitioned that have partitions and stuff like that are um Are somehow semantically related to to like a to something like a correlation ID That's one way of seeing it. So maybe I don't know. We can just say to this guy. Hey use partition. You should use partition key Again, I might I might have said something stupid I mean, I I think I mean From my perspective what you said, I I can understand that that thought process But I could definitely also understand someone looking down and say sure semantically. I It's kind of similar The partition key is for something like Kafka, right? And if I'm not doing coffee make no sense, right? uh a partition key uh a partition key is Yeah, it's for something like Kafka. Yes, that's true. But uh From from a semantic point of view is a correlation event Yeah, like I definitely see that Even if you think it's your key. Okay. Did you get to it? Did we delete the extension that we defined? For partition key, I didn't think so. No no for a correlation Oh for correlation to be actually defined one We had one originally Well not extensions partition No, we don't have one right now So I'm I'm sensing a trap Oh, no, go ahead. What's the trap? The trap is there's a The trap that we should not fall into is that the rpc people get a hold of cloud events and Start doing correlation ideas. Oh, this is the response to this request That's the that's the trap I'm sensing here It's a distinct possibility Yes That goes to the argument that I was trying to make which is okay great, you know Microsoft has this wonderful definition, but it doesn't match At least two others we have come up with that are very popular Microsoft admin has a little bit of a trap. What did what did we do? Well, look at this right here I'm gonna blame you Clemens shared interesting Yeah Oh, that's monitoring Yeah, who cares monitoring That is Correlation idea So why don't we do this? Yeah, so so why don't why don't we do this? Why don't we say look We're not changing our mind at least that as of right now However, if this person feels strongly about it just like anybody else He's welcome to open up a pr to create it as an extension And if for some reason that extension becomes very very popular then we consider consider adding it We can consider adding it to the spec, but as of right now We have no desire to try to come up with a single definition. We failed in the past Yeah Anybody okay everybody okay with that and let's let me turn it around Is there anybody on the call who thinks we should reopen this issue? Okay Since I've been talking with this person I will take the action item then to reply back and Okay Sound okay with everybody? All right. Cool. Um In that case next on the agenda is to go back to our deep dive Designed session But first let me ask are there any other topics people would like to bring up on the call? For the main agenda The main call Okay, in that case, let me just do final world call for the folks who may end up Vanishing since we're going to jump back to the deep dive the other dug. Are you there? Yes All right, cool. And matt are you there? Yes Yes, okay. Cool. Did I miss anybody else for the roll call? Christian's here. Oh, christian Thank you Anybody I've already heard me, but I'm also here close. Yeah, I got you somewhere new didn't I? Oh, no, did I forget to write you down? I could have sworn There you are. It's just not the mark already. There you go. Gotcha. I'm sorry Okay, did I miss anybody? Yeah, I couldn't find myself on the list. Oh, man. Well, you're right. I forgot. Yes Thank you very much All right anybody else All right in that case if you do not care about the deep dive design or you don't care about the discovery interop You are free to go and thank you all for joining Otherwise, we're going to switch back to the deep dive design discussion All right. Thank you everybody for joining All right, so let's see where we Are you? All right back to design Okay, so we already talked about doing an ore. So we talked about creating another dialect Oh, okay. Just for those who joined late Just to bring you up to speed One of the things we talked about Was whether Or was it okay Whether we wanted to support an ore In the filter expression right because right before today's call filters was basically a list of Expression things or whatever these things are called And there's an implicit and because it's an array And we had some discussion and we decided Well, let's make it a singleton instead of an array But however add a grouping dialect so that you can then add in whatever semantics you want Basically an and or and a not and this allows for nesting of these filters as well To give you whatever kind of complicated things you want to do, but that means the bare minimum That people might choose to support would just be if they just support just basic It's no ands. No ors. Just a symbol string matching type thing Okay, so that's how we're going to get Ores into the filter expressions is do this grouping grouping dialect And then the open question was is okay. We got an and an or and a not Should we do an exact one kind of thing or one of thing? And we decided as of right now to not include it but to revisit it later If people can come up with a with a really good use case for why we want it Okay The other thing I want to bring forward is as we had that discussion Clemens that as we got into this sql example Just because um, I wanted to see what it would look like if we actually use the mythical sql The proposal that slinky was working on and what it would look like and it you know, it's a fairly nice compact thing pretty cool However, Clemens said well, what if we From a syntax perspective change it so that instead of Being as verbose as we are what if it looked like this where the dialect type is actually sort of the name of the object itself or the key instead And that's following a similar pattern. We've been seeing in Jason schema when you got stuff like this Okay For those new folks who weren't on the previous part of the call any questions about that I think those are the big things that we talked about All right moving forward then um Let's see. Where were we Okay, so one of the things I wanted to ask about is um Filters right now are basically defined as A conceptually filtering over a stream of events that are coming from the event producer Right, it's that phase two that we talked about where phase one is creating the event Phase two is filtering the events and then phase three is sending so phase two as we currently have it sort of As a mental model is strictly filtering over this stream Which means you're you could only filter on the properties that appear in each event as it's going to appear on the wire Okay, so I started wondering Especially during the previous incantation of this github example of well, what if I don't want to filter on something that It appears in the wire. What if I want to filter on some attribute that's known to the event producer But it's not actually manifested on the wire itself Should filters be allowed to actually Include things that are not in the event itself Okay, and that's the gist of sort of my question for the group here Before I state my opinion. I wanted to sort of stop there and see if that makes any sense and see if there are any opinions on that no one Okay, my Am I thinking about this? I came to the conclusion that we should keep filtering to be just A filtering mechanism on what's on the wire or in the event itself um, if someone wants to support Uh filtering over something that's not on the wire I would be inclined to say fine. You can do that But use the config stuff that we talked about and define your own config property thingy Even if that config property thingy is a filter do your own thing um But let's keep filters pretty Pure and simple and it impacts just what goes on the wire But that that was just my initial thought process What do people think? Somebody's highlighting like crazy All right, see let's plus ones in the group anybody think that we need to expand filters beyond what's on the wire And well, did you want to speak up? No, I agree we shouldn't Because it's the producing stage. It's maybe that's the problem with what filtering it comes on the Subscription on the message stream so it can only apply to messages my opinion All right, okay cool All right, I just wanted to make sure we thought about that. Um Okay, so This isn't actually trying to get to any kind of decision other than As I was thinking about this last night. I thought well, what happens if Someone actually doesn't want to specify source or type and they want to try to specify the entire github A subscription that we've been playing with Using just filters Okay, and what I decided to do is say, okay. Well, what if I choose to write it as well? Here's my normal filter I want just these two types of events But instead of specifying the event source. I'm going to specify It's where there's A property called owner and repo. Actually, this isn't going to work because we just agree we can't filter on things outside the message If you could specify if you could filter on things outside the message This is how I was going to say you someone could do it But since we just killed the idea nevermind. This isn't worth even discussing anymore So I apologize Okay, next topic that I had wait a minute Eric. What are you saying here? um, no so Source and type are filterable Because they appear on the wire Does that answer your question Eric? Yeah, that's good. Okay. Good. Okay. So this this question I have here was actually driven I think it made me clements last week who either asked about it or Thought about the well, how do you know which event is related to which subscription? And I was wondering whether we should consider adding a formal cloud event extension meaning it's an extension not in the spec called subscription ID That way people when they get an event um Can know which subscription is related to so they know exactly which subscription resource they need to to delete to kill it um, and if we would if we did this I would say that we should make it a A should for the subscription manager to include it In the event sense. I would love to make it a must but I'm not sure we can that's why I was thinking of making it a should What if you will thank I have an opinion about this. So if my subscription manager was acting as a third-party subscription facilitator and I have producers not knowing about the subscription handling It's gonna be tough, especially if I have multiple subscriptions onto which I want the Uh, then producer or broker at the subscription ID, which is a featured I wouldn't have with the remember when we talked about the firehose thing adding subscription IDs to all of the events I I don't think that is a good idea So are you saying So I understand you're saying that there are definitely situations where you don't have access to the subscription ID Therefore as a piece of middleware, you can't add it. I definitely understand that which is why I I agree with you. It cannot be a must Do you think it would be a mistake to add it as a should or even a may or even a recommends so that it's it's clear that it's optional But it's recommended. What are your feelings on that? this this Double in case I have multiple subscriptions that give me the same events. Does it double the events? Say say that again. I'm not sure I understood. So, uh if I receive events on a sync from Two separate subscriptions. I these the same event will be replicated and sent the same sync with different subscription IDs, correct? Oh, um, I was not assuming Someone would duplicate the event Um, is it possible for a single event to be associated with more than one subscription? Yes, it would be Because because I think the the event will be um, well You would have to annotate annotate the event That's that's also why it makes make sense to do it as as a as a um as an extension Because you can imagine that if the subscription is is selecting the events To deliver it it could go and annotate the event while delivering it But then if you have cascades of of subscriptions The original subscription would would be lost So that's interesting so so the example that I said it in the first in the in the in the first hour with you know The bulk subscription in the sap thing and then that running through the the the event grid on our end to kind of allows More differentiated Subscriptions will be one of those examples where you have effectively cascading subscriptions on top of each other interesting with would you And and just to add that The the original subscription the bulk subscription that That azure makes on the sap system is nobody's business Because that's a private relationship between The sap system and the azure gateway Right and the user the user is not even supposed to see that that that happening because that's just magic under the covers So that last bit is that are you implying then that you would not want to include this on there? Or that the the middleware would be responsible for removing it The middleware would probably want to drop it Okay, um, and then and then it might go and add In the end I'm not so I'm I'm I'm on the this is probably not worth its size of the fence probably Because it's adding complication okay, well I do appreciate the Oh my god, please unsubscribe me from this list attitude that you have on this one That yeah, that's what it comes to because if I'm getting tons of expense from different github Or different event producers Sometimes it may not be easy to necessarily know which subscription actually created that because Especially if I can create more than one subscription is the same repo kind of a thing, right? um so to to the notion of a single event being Either related or caused by whatever word you want to use in there more than one subscription If this was an array Would that alleviate some of those concerns? So it could be just a list of subscription ideas and Is up to you to figure out which one is actually meaningful? Well, you need you need more than that Because you need you also need to have the subscription manager that that subscription ID is valid in Yeah, that that that correlation I was assuming they they they would um Figure out on their own right because when you mean do the subscribe you get back an ID In the example that I just that I just gave with the the the SAP The subscription and then the azure subscription um the end user will not Be able to reach the first level of subscription manager and not know about its existence True, I I agree with you that there would definitely be cases where the receiver of this may not be able to use this information But I think there are going to be other cases where the person receiving the the event Does have access to these subscriptions or at least they were the one that did the subscribe And so they can have access to the correlation So I granted I agree with you. It's not a hundred percent guarantee That's why I at one point I thought about even creating a URL to the subscription object itself But even that's not necessarily valid because it could be You know a URL that the user may not be able to have access to or the URL They're using doesn't match the URL that the system thinks is people are using right because there's a Some DNS magic going on right that kind of stuff That's why I thought this the ID itself by at least can provide usefulnesses to some people Yeah but I think of this so so I I think this is a As an optional extension That is an annotation of the event which means this is always applied by The events Did this patcher effectively that so so the one that's actually that's actually in the subscription That will go and take an event and stamp it with its own identifier When the event is being delivered then that works okay So you said something also in there in a couple minutes ago or come previous minutes you said something about how Making it an extension and that definitely was my intention. It was not I was not planning on adding it to any of our specs it was just To be able to think it's useful, but I before I actually go off and even think about creating an extension spec Is it worth it? Right? Um, do people think that they actually would make use of it? I'm sorry, Ryan I'm I think I'm a little bit unclear on what is the concrete purpose of it So I think we're like talking about a few things we're talking about being able to unsubscribe Which in that case I would argue um the uh consumer Should just use the subscription ID of the Manager that I used to create the subscription and shouldn't know about sort of the transitive chain of proxies and managers upstream But we also talked about being able to disambiguate duplicate events And also talked about correlation. Um, so I think I'm We should be specific about what exactly this should be used for before we define the rules Okay, well my my initial thought process here was for A way for the receiver of the event To know what subscription it's related to and I did modify that to say which subscriptions It's related to but I was originally thinking of singleton Because it's possible that I can have more than one subscription for the same event producer Such that I could get multiple events from that Um for that event producer and not be able to uniquely know exactly Which subscription caused this event to be sent to me? And so having the subscription ID allows me to make that correlation to say I don't want to randomly pick one I'm gonna get get it wrong. I want to know exactly what subscription I need to delete to stop this event from ever coming to me again Is that help ryan? I think I need to think about it Okay John your hands up Yeah, so I guess uh I I had the sort of the same the same question about the the specifics of the use case but but also adding the the perspective of What additional burden are we then putting on each of these subscription managers in terms of? You know tracking state About these subscriptions, especially as you were just talking about You know with overlaps and You know aggregate proxy Situations where they have a fire hose and they're trying to pass that stuff through as as cheaply and quickly as possible right, are they like what's the what's the What's the memory and processing burden that they're going to have to add on top to be able to support this And then we're talking about use cases that it's going to be hard to even support ambiguity and things like that so Yeah, I want to understand more about specific use cases where we think it's The only way or the best way or somehow more efficient to put that burden on the On those aggregators those proxies versus the subscriber having to deal with their own subscriptions Okay, thank you. Uh christoff your hands up Yeah, I'm also not sure on on the use case and I think the whole idea that you just okay. I don't want this event I'm just unsubscribing If you're not so sure what your subscription do in the first place and you made the leader subscription It actually sends you I don't know five or ten different event types We're not interested in this one particular event, but then you're dropping all of those event types So I think in all cases you really need to look at your subscriptions in detail Before you can just start dropping things. So maybe the correct operation is not deleting it about modifying the filter To get to filter some things out Yeah, it's interesting and I was also reading your comment in the chat Where you said it should be easy to figure out based on the type and the source and I agree if you're maybe If you subscribe to a very specific thing I would agree But if you if you basically said You know any source any type I'm just going to subscribe to this to this particular subscription manager Then it becomes a little bit more challenging But I'm not I'm hearing enough people questioning it that I'm going to I'll go off and think about it But as of right now, I'm not going to take any action because I'm not hearing widespread support And if there's not widespread support for it yet, then that's probably not worth it So I think I got my answer Which is simply no not right now Which is fine Okay, so we need to make we don't need to take up more time with that. I'll just let us sit Okay, let me just see here Okay In that case that was the end of the github example um Now clemens you and I talked earlier today Did you get a chance to Start thinking about or writing down what A subscribe might look like in a in another example like a mqp or something like that Or mqtc I did not I I did not have time to write it down, but I thought about this a bit and and there is a Like in the subscriptions documents There is a Section that basically says we don't want to go we don't want to go in and and invent new api for I think on top um Yeah, I came across that today too. Where is that mqtc? There here And more But this isn't it You know, there's a there's a I have a section on this on subscription and Effectively describing the the inherent mechanisms of some of the Um Yeah, so there's exactly so there's this consumer solicited delivery the pull style And then there's the push style the push style Delivery and I think those are different In terms of how we think about those HCP doesn't have a mechanism doesn't so this might get long hang on Because I want to say this now in my own words other than just having you read it So I'm not gonna look at it. There is there is There's one way where you when you walk up to a subscription manager and say hey, please give me events Here's an endpoint That's what we discussed so far And and all the things that you have in your examples are true for all of the different channels um, htp and mqtt and and kafka and mqp if the delivery is push meaning if The delivery is initiated by the Subscription manager or the the delivery agent of the subscription manager meaning The delivery manager Actively opens up a connection and then starts pushing events through that that's that's push delivery the subscription manager initiates the the the delivery um, that is suitable for all the scenarios where you have serverless We still are here scaled to zero scenarios Where the resource is dormant Again, there's only some end point which is listening and then you open up a connection into that end point You start pushing events to it and then the machinery On the other side wakes up loads the required code and then this this patches that event. That's that's why push delivery is So attractive for for serverless. It also allows you to go and do you know classic htp style um traffic balancing load balancing Um, because you can basically just go meter the number of requests that you're getting and then start start up new new resources So push delivery is great for that Pulled delivery is more what you do in in classic messaging system where you are effectively hanging on a queue Or you're hanging on a kafka topic and you continuously start pulling events out For that mechanism, we don't have to go and invent anything We shouldn't invent anything but we should really go and just just say use the mechanism that's in your mqtt broker Which means if we have a if we're pushing events into um an mqtt broker Then the gesture to subscribe to those events is exactly that of mqtt, which means you use the mqtt subscribe um gesture The the the protocol gesture And and and then on the topic where you subscribed in mqtt The events that you that you want will come out And then the question is there. How do we how do we? Map some of the constructs That we have here like a filter and capabilities, etc. How do we map those onto mqtt and they will be constrained naturally by the capabilities of mqtt because you're choosing the protocol and so you have a constrained set of of ways One way or the other what you're choosing of what the broker already gives you And in mqtt, that's the same thing. You create a you create a node. The node has a distribution policy Of copy that is equivalent to a topic in mqtt and you open the link into it and then messages flow out In kafka, you create a consumer group Um implicitly you attach to the consumer group and once you have a consumer group you get effectively a um um A copy of your own stream or your own copy of the stream Where you can then make progress on that stream independent from from other Consumers which make progress that it's not exactly pops up, but is is is close to it So the the notion here of this section is to say let's not invent new ways to do pops up with protocols that already have pops up built in If you're doing pull style delivery and that is why This document focuses on how to set up push style delivery And then it's effectively saying if you're using mqtt, if you're using kafka, if you're using mqtt Well use the pops up mechanism that's in the protocol And that is what that does and that's too obviously So this is what this is how and and there i'm effectively just uh describing how that works so Everything you said makes sense to me. I'm just trying to figure out What that means in terms of the spec for these pull style things because obviously then there's not a Subscribe type operation we're defining per se No, we're saying hey go use the native stuff. Yeah, we're using what's there. Yeah, exactly because I mean What we're trying to do here is we try to facilitate interop, right? and And if you're using mqtt, well the interop is defined because you're using mqtt use that protocol go read the spec and implement it Right, so do we need to actually do anything for example? Do we need to say hey for mqtt? uh, here's how the concepts of what we've defined like a filter mechanism or the The the config thing Those concepts would map to these things in the mqtt world. Do we need to say that or is it not necessary? uh, i'm i i specifically made this the sentence above the mqtt says it's non normative I'm basically just mentioning those because Because it's I think it's not up to us to go and um um You know configure define what a broker what a broker should look like And I think it's not up to us and also not in our powers really To go and say Kafka should go in and deal with pop sub in the following way Just because the events are cloud events Um, or or mqtt or mqtt should do it in this particular way where we can go where we can Influence things is is in this push model Because they're effectively the the description manager acts as a client and There is that's that's why we have our own You know pops up mechanism kind of under our own control if you will But here if we're if we're using an mqtt broker, which is built for That kind of distribution pattern will just have to go and use what's there in the protocol Okay, class your Oh, sorry. Are you done clements right now? Okay. No, sorry. Okay class your hands up Yes, so in general I Completely agree to clements. Um, we just have some edge case and might be out of scope for the specification here, but um, in some cases you might still want to have this, uh, subscription api based on cloud events because the the other end is In addition to just I mean is is creating actually some endpoint For example for a mqtt let's say a queue And and configuring this so you would then As a result of this subscription api call Get something like a queue Configured and then do again the pull style consumption on that Um, I'm not sure how that fits here And that I think that would be that would be in the um There would be an option in the push style the delivery model So it's it's it I think that is a that would be one of the protocol options Um, if we wanted to add this in for mqp where you say or for mqtt where you say um Deliver into this topic, but then or deliver into this queue and if the queue doesn't exist go make it Which is I don't know if it's just wait clements. I don't know if it's just me but you cut out there for about five seconds Okay, so so this one might be an option in the an option first in the mqp Protocol settings and in the mqtt protocol settings where you say So in mqtt it's actually implicit Where you say, um, you know if the queue doesn't exist go and create one okay And then and then and but that's that is that is the end of it That's the end of that special configuration kind of on the cloud events side because then you made a dynamic node in the mqp broker And then you you look at the mqp broker from the other side of the other side is just mqp And it's same industry for mqtt, right? So you go and and if you just specify your mqtt endpoint and then you specify a topic path The topic path comes into being an mqtt And then you on the other side you just go and start consuming with normal mqtt The mechanisms without having to consider anything special as for cloud events And the only thing is you're getting you're getting messages out of the the mqtt topic That are mapped by the rules of cloud events But how you obtain them from from the mqtt broker is completely defined within the mqtt Spec, it's not something that we you know the question here for those for those Documents that are writing here is do we is there anything missing in mqtt? Is there anything missing in the mqp to get at those messages and the answer is no It's a man. Well your hands up Yes, um, I'm not sure if we have a common understanding on this topic because I'm looking at the sample subscribe You put for the Kafka subscription manager and there it says consumer group as a config parameter And also from what I heard from Klaus it sounded as if the subscription Creates some sort of a queue for the subscriber to consume from And this is I think a misunderstanding because even though it is called the Pull style protocol model and the delivery of These brokers is pull style to consumers always because the consumer of course Connects to the broker and then pulls messages out of it. But as Clemens described it, this is a consumer solicited Model in which the consumer provides the broker And then in the subscription Advises the subscription manager to create or facilitate this Subscription in which produces sent to the consumer provided broker So as you wouldn't have a consumer group to consume to publish to what the what the producer does It publishes to the broker that has been named like the sink that has been pointed By the subscriber Yeah, you are absolutely correct. There are two things here. So so the example that that is here That we're looking at is wrong Because we will be only configuring to send to a so this will be effectively setting up a push delivery into a Kafka That's what we would set up with it with an htp call And we would not be setting up it like if the subscription manager were a Kafka broker We would not be using that gesture. We would just simply be attaching to the consumer group and and and use Kafka native mechanisms. So that's right. So that's the first thing the second thing that we said, so so so wait. Okay, wait Wait, wait, wait, just let you know First of all, I don't doubt everything I have here is wrong. However Keep in mind. This was not written with I This was written with k native in mind In the sense that if if I wanted to subscribe to in essence a k native Or I want to create a k native event source Which that event source is going to be the one that's going to be pulling from a Kafka But then delivering things to me over htp to my event sink I need to tell that event source how to pull the events and where to get them from That's where this information is coming from and I was trying to figure out how in a k native world, this would look So I think it's a different scenario than what you guys are talking about. Okay, great. So, okay. So, okay, I understand that now that makes That makes that a little confusing But I understand that now. Yes Yeah, you're you're you're not throwing in some Some Some overlapping concepts because this is what you have in conflict here is completely a concern of k native and something that nobody This is an internal implementation detail Kind of All right k native world today when I say hey, I want to get events from Kafka I set up an event source and I and I basically give it this information Yeah, okay, but then I have to tell it where to send it to and that's what those here So this is the exact equivalent of a k native event source. I just put it into our syntax Okay. Okay. So that then then I understand that yes so that was the so so if you were just if If there are if you're using Kafka as your pop-up engine And and what we do with the subscriptions API is we want to we want to facilitate Subscription management effectively right for in an upper interoperable fashion And if your subscription engine is an NQP broker or an NQTT broker or Kafka Then Well, you have a subscription you have a you have a pop-up engine right there And if you if you are building this with REST HTTP As we're doing here, then we need to go and effectively add the the the pop-up engine if you will to As a as a thing and we need to make sure that the the pop-up engine is interoperable That's what we're doing here. So that's the the But we don't need to do this for for for pop-up engines, which people already have that was my Intent here in terms of of What we just discussed earlier With you know being able to create a queue on the fly and then deliver into it Um, that's something that is that you do implicitly an NQTT and that you may also do an NQP is that If you allow that any entities then you can go and deliver into something that doesn't exist And then the the the queue or the topic will then come into being So so yes the the subscription can cause a queue to appear That hasn't appeared yet, but the broker obviously need to exist So let me ask you something Clement or actually Manuel your hand is still up. Is just have a question or is that old? No, um, sorry. Okay, that's fine So Clement's a massive question In terms of the specification itself It seems to me that when we come to pull style event delivery, right? The fact that we even talk about MQTT and NATS and all their stuff Is interesting, but it really has no effect on the spec itself, right? Because I think what you're basically saying is Our spec is all about push style delivery because any kind of pull style delivery. Well, they already have it defined. We don't need to touch it Yeah, the reason the reason I put that in here is That I know how this goes and and people will say Oh Now I have to go and build all this complicated shit around my my Kafka broker to be compliant with cloud events So so I want I want I want to make make it clear that our intent that the intent of this group is No, no, no, don't do this use the Kafka mechanisms. They're fine And this is that that's that this is effectively in the smack in the middle of the spec a Clear hint that you don't have to go and construct any of that stuff if you already have Have a pop-up engine and if you're fine with using MQT MQP and Kafka protocols If you want if if you're all this htp rest and that's all you do And you're and you're living in an rpc world while then you have problems with directionality Here's the way how you can go and do all those things So I have it I mean with with cloud events We gave people the choice of what five protocols right gnats mqp mqt Kafka and htp and For how to go and map those things and and four of those have native Pops up mechanisms while htp does not And so what we're doing is if we're effectively making we're making the htp world, which is The biggest the biggest world of them all We're we're we're helping that world to kind of come to terms with pups up But then at the same time we're saying let's not invent top pups up for the other worlds That's that's the motivation for having that here because because I can bet That if we don't include this if we don't make that clear That all all kinds of folks are going to build weird contraptions around these brokers Just to be compliant with with cloud events and that's what I want to avoid Okay, so technically this none of the text about poll needs to be in the spec But you're doing it as a pre-emptive strike and I'm okay with that Is it totally it's just a preemptive strike because I It's I'm sure That this is going to this is going to turn out badly if we don't make it clear that that's not our intent that you go And build all the rpc stuff around those those brokers Okay, so criss-toff your hands up. I was going to ask anybody has any different opinion on this So criss-toff your hands up I don't necessarily have an opinion, but I have a question for clements So, uh, so for our company what we sometimes want to do is build integrations. Um, that are basically event consumers They're open source. So people should Install them in their own system run them maybe modify them to make fit for their use case And then depending on who the customers they usually have their Things set up. So they may have Kafka. Um, they may use something from the cop writer So that they're kind of set up already. So we think cloud events would come in there really well to Yeah abstract over those layers um for us. So but in in that case also part of this Is that we know what to subscribe to so that would always be um the same so if we had So basically what i'm saying all this what you would you label all the shit around Kafka and so on That's actually what would be useful for this case. Um, but what I hear you say is we I don't buy that argument you know why because you set up a Kafka server to Do 10,000 100,000 events a second That is very different from uh being called 100 times a second On an htp endpoint. It's just a different story like all of so event flows What the reason why we're having all these different kinds of infrastructures? Why there there is nqtt and why there is nqp and why there's Kafka and where there's so great competition in this space Is because there's tons and tons of different tiny little use cases That caused those protocols to exist that also caused the the variety of different products to exist And the fact and and being able to go and and and and wrap them all with the same protocol with the same api Is something I find That is impossible. I mean there are already those different protocols And having the same management api around them is similarly impossible because The the the setup of these brokers are also is also different right the nqtt broker from topology perspective Is totally different from a Kafka from a Kafka broker Well, I think what we what I what I care about um, and of course, that's just our Why our perspective is that the events look the same that you can go and tap an event hose And the events that are coming out are of the same format But I don't think we are in a place from an architectural perspective that the event hoses can look the same right Yeah, I mean that's not necessarily what I said So what I what I'm seeing in companies is that they really specify on one thing and then they want everything To consume we are that mechanism. So they say we're a Kafka shop You do everything Kafka and then if that's the best protocol or not The for this particular instance doesn't matter, but it's the thing they just use internally Well, maybe it's a different And those folks show up at my door every day and I tell him that's stupid I mean Okay, well, you're in a position to say that I'm not so Yeah But but for that case, but I mean if if you have a customer Who says, okay, we're doing everything Kafka You're okay because you have a common way of how how to express events And then you only have one api And one mechanism for how you get those events and that is Kafka And and with this document as it stands You'll be okay with cloud events because we say Kafka is a fine way to go deliver cloud events You'll be okay Right, but basically we still have to do for each For example, Kafka, we have to say this is how you subscribe And then we have to do the same for each other Protocol Okay, I got it. You can't I would love for us to be able to go and fix that here But we can't even agree on Yet on common on common apis That are polyglot and work across all kinds of of programming languages Just for a queue Right, that's that's That is already really complicated and we can't like we can't agree on on those things for for something like like Kafka because For Kafka the patchy Kafka project Equals equals confluence. They're running away at the they're running ahead with with whatever the Kafka API is and then there is You know pulsar and there's Azure event hubs and there's kinesis and they all look different And having a common api across all of them is something that the spring people try with spring miss with spring cloud stream where you already have a bunch of Works and differences across them, but then that's only for java And and there's no such effort for c sharp There's some of that which mimics what's happening in spring and go There's very little to nothing in javascript So, I mean that's that's a problem that I would like to solve at some point down the road somehow But we're just not at that place Okay, uh, thanks for your thoughts. I think We can discuss this forever, but thanks. Yeah, no, so I just realized that technically we're 10 minutes over for the discovery interrupt call So that's just quickly Does anybody have any topics they want to bring up for the discovery interrupt? I suspect everybody is either busy or Clemens. You're still coding away But is there anything people want to bring up? Okay. I just want to miss out on that If someone joined the call just specifically for that. So then circling back Um, you guys can ignore this. This is just my own ramblings And I just I actually more want to get like scott's point of view on this more than anything else um That was it in terms of what I wanted to walk through in terms of a concrete example To to try to force us to have just some discussions and go with some design points And I think we came up with a whole bunch of things that I guess I I have the action that creates some prs for these things to probably get them in the spec Are there other scenarios that people would like to walk through either today or next week? um That would warrant us needing another three-hour phone call or starting next week should we go back to our normal one-hour phone call I'm not hearing anything. I'm going to take silence as Let's process this through our normal process and And kill the three-hour phone calls until they're needed again By hearing that right Yes You sound hesitant. Did you have something in mind you think we need to deep dive on? No, um, we we can Not next week Okay, I mean obviously we can resurrect it as necessary Okay So as of right now any objection then to canceling the three-hour phone call for next week and going back to our regular one-hour one Okay um And like I said, I'll take the action item to write up the prs or issues or whatever around these things um and and see how we how we like it once it's actually in pr form um Are there any other topics people want to discuss at all? Otherwise we can end early today Christoph I assume your hand is old, right? Sorry, yes Yeah, Manuel did you want to say something? Yeah, you don't want to ruin your day by extending this but um so when you raised I don't know. Is it okay to have another half an hour or three quarters design discussion up to top of the hour as planned or sure. Yeah, of course. That's that's why we're here. Well, please. Okay no, no, no I think you raised a very interesting point was the example you gave was the k-native subscription so Until now I was under the impression that configuration is really only about The subscription towards the consumer so that always the subscription because if I look up an event or let's say a service in the discovery And I get the subscription URL to which I point my subscription I only tell it how I want the events to be delivered if it supports HTTP I can have this push style delivery to my consumers and if it's Kafka Then I know it can produce into my Kafka cluster but The configuration you gave here telling the subscription manager how to obtain the events from another source That was a bit baffling to me. So That's what it was intended, right? I overlooked that the protocol selection was HTTP and then you gave a Kafka configuration So this would To me sounds like a very generic subscription manager. I could more like a An adapter really where I tell okay, do subscribe to there and deliver everything by HTTP or the other way around Is that even a real case? Is this part of the subscriptions API? some I so I think the way you described it is the way I had it in my mind because to me That's exactly what k-native does right these little event sources that you can create in k-native To me, they're like adapters, right? It's saying hey go manage This event producer for me or talking to this event producer for me and in this particular case The event producer or that the event producer But the thing where I'm going to get the events from is a Kafka queue, right? And so This is this is how I want the k-native infrastructure to Talk to Kafka for me, but once it gets those events, it's going to send it to me over HTTP So yes, it is like an adapter um And whether it makes sense or not to set up this adapter via a Subscription thing. I don't know. I just thought it would be interesting to see what it would look like if you tried to model it that way um, and If we're going to model it that way I couldn't think of any other way to specify this three bits of information So that that's how I end up where I did So in a classical sense, it would be a subscription to a Kafka Oh a sorry subscription to a subscription manager that can produce Events through a Kafka and Channel and then you I would point it to my Little events or my k-native Does it come with the broker? That's a question. I mean it's it's k-native is really it can only consume And then give the events deliver the events as an HTTP push, but It's not operating the Kafka cluster. Isn't that right? Unless the channels I mean talking about the source specifically Yeah, yeah, ignoring channels. I agree with you k-native is not managing the Kafka It's not setting up a Kafka cluster and like that. It's simply Consuming events from a Kafka cluster through a poll model. Yes, and this is configuring that poll mechanism. That's all it's doing So then my question to Clemens would be since we have the consumer solicited a poll style and the push style Is there a certain thing where There is a poll style that is not consumer solicited Uh, I No, there is not because those it's meant to be synonym It's we have the reason the reason why this why this a little bit High nose term consumer solicited exists is because we are already had disputes about the terms pull style and push style And so therefore I I use that as a clarification term that someone is initiated Inward initiating the delivery and To make clear what what pull and push means So, yes, it's it's Customer a consumer solicited is pull and that is meant to be the same thing Okay, it's my understanding then correct that the broker Address is still something identified by the sync and it's something that is provided by the consumer Yes, so so this is why this is why I was this is why I was um Telling don't this might be confusing because you are being confused by it The Kafka the Kafka thing here in this in this very example You should ignore this because it has nothing to do with Kafka it is simply about how The implementation of this particular subscription manager Turns around on its backside and goes and grabs events from Kafka that it will then go and push deliver to this htp endpoint From a cloud events perspective the config that that that you see here Has no function whatsoever. This is purely just to inform some internal implementation of how to go and get get some events This could just as well be an opc way the configuration I I still doubt this falls under under the subscription. Well api Yeah, so to me this case doesn't fall under under the subscriptions api entirely I Think my internet connection is going bad. Can you repeat that? Yeah, it's I think this case is is not for the subscriptions api well Let's put it this way I I would agree with you that this is Probably not a driving use case to define our specification. Okay um, however I think This is a valid use of our specification if someone chose to do it that way All right, because there's nothing in here that's illegal according to the spec Yeah, so when you're coming from discovery, there would be so today in k native I'm sorry to say it again So from this coming from discovery, I'd have the config that tells me service topics and consumer group are three valid fields And it tells me the type of these fields and then but What is written in there? This is entirely up to this specific Subscription manager correct So it is valid in a sense. Yeah every yeah, yes, that's my point Well, I would guess work is a little too strong. I agree with you. It's it's It is specific to this subscription manager The definition of the config section though is up to the subscription manager, right? Because even in Even in our github case, but we don't have any config anymore, right? But let's say there was a config even in github, right? In order to understand that you have to understand the discovery Entry for github to know what those config values mean, right? Because the discovery endpoint Let me see if I can find it here The discovery endpoint gives you the name And it gives you the type Right, but that's it to understand what that actually means you have to go manually go look at the specification for that field To understand. Oh The word org means github organization right So it's not something you can programmatically Fill in Right unless this unless you happen to know in advance. Oh the url points to this thing and and therefore I know what it means, right? But if it's a brand new url, yeah as a person, you're going to have to go read it and go understand What it means to use this particular this particular name inside config and so And so this is the exact same example, right in order for the user to understand how to fill in servers They have to go check the specification for this subscription manager To know how to fill that in and to know. Oh servers in this case means. Where are the Kafka servers? Does that make any sense at all? Agreed. I I guess that's my background trying to discover services and subscription urls that I could Pull or I think get events from In an automated fashion But subscription config Is is just as you said it is specific per service and requires developer work um to actually consume These events correct. Yes And I think I think that's always been the intention because every subscription manager may have their own unique fields that you can Fill in either optionally or maybe some of them are even required Right, that's up to the subscription manager to decide That's why I understand when you look at this you may say, oh my gosh, this is just a weird example But I do think it's perfectly legal Yeah Agreed. Yeah, okay All right. Well, there are other things you wanted to mention since you said I don't want to take up 45 minutes No, not the entire time hopefully But there is something and if I may just temporarily I Paced it under Clemens's edition A little bit on the top. Oh now it comes with vs code coloring. Sorry for that That's this thing right here Yeah, not even shows the syntax highlighting. I'm seeing but okay, um So I had a look at this flattening of the dialect and when you compare the first two in all It to me, it doesn't really add a lot. So I Of clarity to The specification because we're adding a nested object where we had just two parameters In parallel And then my question following that Optimization I agree it reads nicely so you can read sql condition type equals and so on If we wanted to do the same for the basic filter We currently we were doing this with dialect basic as I pointed out here and then to flatten this We could say basic and then still have our three Properties there type property value Or we could go one step further and then flatten the basic Types in which case we would end up with Please go down a bit If we said okay basic is just an indirection of specifying one of the three matching types exact prefix and suffix We could flatten it even further and say take the Property that we want to compare As the parameter name so here type And put the value In the value section of that so we can have an exact matching on type Come get up issue or a prefix equivalent with come get up and the suffix equivalent so it was just an idea if we Do a little bit of simplification and flatten this dialect into a property name, but at the complexity of having a nested object To me this doesn't give much back to this back But if we went one step further with the exact prefix and suffix This would make it Yeah more syntax less text What do people think about this I mean to me If you to me if you buy into this whole notion of putting all here This is a natural next step to be honest. Yeah, so yeah That is true. Well, if we if we can find how If we can find a get great way to escape effectively other dialects, and that's great It's so exact and prefix and suffix become their own dialects That's actually not terrible That's that's we have an exact filter a prefix filter a suffix filter in a sequence a sequel filter So effectively we're now Which which mirrors effectively in a computer model Where you don't have you don't deal in dialects, but you're dealing individual filters And then you can just go to compose the filters. I'm fine with that It's interesting Great. Just wanted to bring it up And yeah To be considered that's all Yeah, no bad idea. It's good Yes, I'll tell you what since I took the action and it's sort of right up what what the old versus new format would look like I'll I'll go, you know To the to the extreme that you're showing here and and see what people think It's a very interesting way to go I like it Okay That's cool Anything else people want to bring up or Manuel? Did you have anything else? Yeah, go ahead Eric So do you have any examples in this of subscriptions where you're not expressing Configuration over a single hop And and to be a little more explicit about what I mean We've been talking for a long time about this potential multi-metalware delivery scenario And being able to have, you know, multiple protocols switched in and out And and I'm looking at the config sections that you've been declaring They seem to be pretty much implying Exactly what a single and uniform path That the subscription Is delivered through and I And that's it's probably poorly stated but bear with me for a second If if The the reality is far more complicated behind the scenes but I want to Receive some set of events through some unknown complicated perhaps Inconsistent i.e sometimes some events come through one Set of technologies while another of the events come through another set of technology I don't know I'm I'm being kind of hand wavy, but I think these are the harder cases Something that as a user I want to be able to specify Is something like I would like the An order guarantee over the events that the order they were Uh That they are recorded in the upstream sources is the order that I will see them as a downstream consumer um Anyway, uh, so the the question is Do we have do you have any examples where you're looking at scenarios like this because I think they may be using destructive for bringing up some Important and You know ongoing design decisions So in that particular case for like the guaranteed order delivery, what's not clear to me Is whether that would be a config thing versus a protocol setting thing Does anybody else have an opinion on that because I think that would have to be part of the answer you're looking for wind it I'm still holding all of exactly where things are a little loosely Um, it still seems to me that a lot of what you're declaring in config is is really a filter and that the source and types are themselves filters, but Anyway The the kind of considerations and concerns that are being properly expressed In that are that are really the real kind of expressing the real needs of the subscriber I think that making sure that those are present present and accounted for Is is of a higher priority Than how we factor it all in there I'm not quite sure I didn't answer you Good because let me let me Let me try to say what was right through my head. So a while ago a couple weeks ago when I was Basically starting to write up this document My mind jumped to I think where you're what you're kind of poking on which is this idea of trying to categorize things Is is a little annoying in some ways, right? Um, and and I kind of then Took that that thought process to an extreme and basically said well Everything's just a top-level thing Right, and you just basically specify A whole bunch of different properties just to tell the the the event manager or the subscription manager What you want to have happen, right? And whether it's a protocol thing versus a filter thing versus a config thing They don't know they don't care It just you know, there's a there's a property defined that you need to fill in to says hey make this semantic happen Right and trying to categorize things was kind of silly the same way We got rid of buckets entirely in Cod events, right? Because I argued it was silly to try to To define A foo in one bucket to be telling different to mean the different thing from a foo in a second bucket Hey, that was just going to be really confusing. So why not move everything at top level? And just pick a better name than foo that way. It's it's more descriptive Um, and I kind of wondering whether you're poking on the same thing here, which is You know, why are we having all these buckets? All right, is that kind of where you're headed with all this? I I think this is the the much less important point Whether whether source and types are elevated into a top level or part of the filters. I mean There's there's fundamentally a specification of what information you want to receive And that's and but there's also a question of how do I in the very last hop want to receive it? And I I think the thing that I'm asking about Trying to be more useful because I'm I'm bringing a lot to the table that's very specific to me and I apologize for that but There's an important consideration of there is a lot that can happen During the original production and then delivery to me in that last hop and As a consumer of events I may Personally, I have some various strong opinions about what I want to be guaranteed Through that entire delivery chain now you know my bias here is I'm really kind of obsessive about order order is equivalent to consensus and You can achieve it without any consensus algorithm or coordination if you can maintain order throughout the entire system And where there are, you know Uncorrelated orders you create a join of event streams in order to main to establish causal order between events and and therefore, you know Anyway, this is the sort of stuff. I I spend a lot of brain power thinking about because I really like the notion it ticks my brain, but That means that I have really strong opinions about what I want from a subscription And particularly that I wanted to maintain give me an order guarantee a guarantee that the order originally observed will be maintained So, you know, this is why Clemens I use event hub and hate What is it the other one the The ones you guys always a great grid you push a lot or at least the marketing pushes a lot, but um I I use that because of this order guarantee and the way that that drives how my systems behave um and and While I can specify the last hop cloud events allows in or at least what I'm seeing specified here in the last hop Uh cloud events allows for the specification of a whole arbitrary or the non knowledge of a whole arbitrary chain of middlewares deliver Delivering from the production of the event to to me And I and I'm I'm not seeing anything that is trying to specify And therefore kind of draw out any of the issues that might come with those sorts of scenarios And the the user demands for them So, um, you know, we're I'm being very indulgent here Uh, the real thing is, you know, the question was, you know, do you have any of those scenarios here because I think they'll be instructive for the sorts of things that As we define cloud events and thought about it and talked about it in the past that we should probably Bring to bear because they really show up in the subscription and delivery But I think the short answer to the question is no I personally don't have any I haven't written up any scenarios to talk about that, but I but I think in general what you're basically talking about is where How would you express your lack of utter phrase quality of service requirements? Is that fair? I think this is a reasonable thing. I I would expect it to go into config personally, but Okay, but so I guess I'm trying to I'm trying to figure out the end goal of your Of your topic Right. Is it that you think maybe this is the the question is to say this I think this is a really valuable exercise that these examples have had Caused us to ask some important concrete questions and bring some of the fine distinctions to bear and um I was saying kind of yes, and I think this Uh, kind of more advanced case might be something that Needs to be considered Okay. Yeah, and I think that goes to what I was asking earlier, which is hey guys Think of other examples that we want to walk through and I think The ones that are that are rattling around in your head are great examples. So if you can Write so it take take one of these samples that you were just talking about right and write that down Into a doc and say this is how you would you would model it right? This is what you would expect your subscribe to look like to get the thing that you're just describing But you know guaranteed order delivery, whatever it might be And let's walk through it And see what falls out of that Maybe it will be a very short discussion because what you write down is exactly the way everybody else would have done it it makes perfect sense or We end up in a rattle discussion like we have for the last two weeks or three weeks and we have to make some design change to the spec So I would love that exercise It sounds like I'm not getting out of writing that user You can pace up. I started doing it last uh, we can ate most of my day and had an awkward stand-up, but I guess I'm doing some again No, I think somebody else needs to take the pen for a while besides me So, yeah, but it's fairly fair Yeah, I mean my When it comes to people like yourself Clemens class, you know, you guys I think you guys have far more experience in this space than I do Right, that's why I stuck with github because that's a very simple example that I personally use on a daily basis So I'm looking for you guys to come up with more real world scenario or other real world scenarios because obviously github is real But other more serious complicated examples to walk through to make sure that this thing's actually going to work so so Klaus and I will have to go and see how much we I'm gonna I'm gonna speak for klaus here, even though he might just be screaming behind the mute I speak up when I have to object So so we have a I think we have a pretty complicated scenario and We will be happy to talk about this With as much detail as we can obviously there is some There will be some stuff behind the curtain, which is not maybe not necessarily all pretty Um, but but in terms of how the cloud events flow Works and how we are using the specs To make sure that we can go and coordinate this. We are both pretty Well committed to make sure that that all happens in the standard way Um, I think we're both realistic about having to cut some corners Given that we have to go and strip something by whatever. I think august Um, but um, we will certainly be able to go and tell you, um Some what we're doing with the specs are for instance the discovery documents already can tell you Um, there will be and that's something we discussed yesterday My assumption is that there will be discovery documents. Um, And when I'm in document, it's the get all from the discovery service Um Will be created somehow in the SAP system Then we will get it somehow and then we will go and take that same discovery document and partially rewrite it and and and republish it Um, but I can't tell you what to see what the details of those are yet But but the the the scenario that we have is we have a large SAS system One of the largest SAS systems Um, which wants to go and publish events and we need to go and turn that into a fire hose of sorts And then split that fire hose up by dispatching it into into azure subscriptions And and the mechanics for that are all going to be based on the specs that we're developing here So we're going to share as much as we can And ultimately that the decision of what's secret and what's not it's really up to Like there's nobody There are no higher level decision makers who who put the lid on it. So we're it's mostly up to us And now sharing's we're sharing whatever you can sounds good to me Yeah Yeah, so Eric back to your thing if you if you can you know write up a scenario that we can walk through to make sure it's Make sense and satisfies your use cases Um and leads to more discussions about whether the spec is right or wrong. I think that'd be good Sure Okay Anything else people want to bring up? All right in that case just to reconfirm We'll cancel the three hour marathon session next week and we'll resurrect or we'll cancel it going forward And we'll resurrect it as needed based upon, you know, whatever design talks people want to bring forward But at least I got what I wanted out of it, which is deeper to find this deeper discussion around stuff like this So thank you all for joining And I'll get her removed from the calendar all right Thank you everybody for joining and we'll talk again next week All right