 Hey Mark. Hello. Good day. Hey Alex. Okay great. So I'll update the meeting minutes with your attendance. That'll be helpful so we can get an idea of who's rocking up. Cool. Well welcome everybody to the UDPA working group meeting. We've kind of had a hiatus recently so there hasn't been a ton of activity in UDPA lands of the last quarter essentially. So that kind of grew my goals of the meeting. I was just to sort of bring those up to date with what the roadmap looks like, what our plans are for Q1 and beyond in 2020. So I think maybe a good way to do some of these to go through the Omicron slide and I won't go through the whole talk just to look at some of the roadmap slides. Is our folks generally familiar with that talk or is this new to folks in the meeting? I've skimmed through the slides. They seem very informative so. Okay so I'll be reasonably quick then when this I gave a talk at Omicron based talking about the roadmap for Omois APIs and beyond which is essentially UDPA. So the basic stuff I'm sure everybody's familiar with so let's just import and just like look at where we are on the roadmap which is essentially where I'm roughly at the end of the last quarter. So we had, you know, Omois V2, XCS APIs, V3 which is what most of the work in APIs that I've been focused on has been around especially in land and intercept the Omboi 1 or 13 release which will be next week where there's going to be a turn right now in the Omboi code base. It's largely a mechanical change but it also puts in place what's called the stable API versioning policy where we revenue major version of the API each year in Omboi and gives us the opportunity to maintain trading support for two versions for basically two years and that gives us the ability to you know make major transactions in the API which includes UDPA. So that's actually happening and is landing and now the focus is on this GPV for which slash UDPA. UDPA is what we're in business of doing in this working group. So yes, I talked a little bit about the versioning goals. One interesting thing to point out is just the sheer scale of what we're doing with APIs here. I'm not sure what the APIs or other proxies looks like like HAProxy or Nginx but you know if we look at the just the sheer line counter the protos which are you know it's a specification for the API for Omboi. It's really really large largely because of the extensions. Each extension has you know its own configuration API but we're talking about 15,000 lines of protos and it's actually more like 10,000 code references in Omboi. So there's APIs are huge artifacts and actually evolving them is required us in v3 to build a lot of tooling to support that and I anticipate UDPA as it grows we'll need some of this tooling at first though where we're taking that it's happy that we'll start with smaller and then grow that upwards. So let's see okay so we're kind of roughly at this point where we're talking about the idea is that you know Omboi is before release at the end of the year 2020 and this is where we hope that a lot of UDPA stuff will intercept and we'll be able to move some of the things that exist today in Omboi off to UDPA and consume that as a first class product and basis for API. So let's talk about what we actually want to get done in 2020. So one of the things we've even done in v3 is we've actually started to let's see we've started to populate the UDPA tree and some of the protocols that we're using the Omboi APIs with here there's nothing here that should be particularly controversial or require massive review there's a bunch of annotations we use to express versioning concerns or the fact that some fields might be redacted or this kind of thing. These are all very structural things that actually make a difference what the API looks like we introduced a new type which is called type struct it looks a lot like any but it allows us to use JSON representations for extension objects in the API but also convey a bit of type information so it's essentially type JSON and that's actually very useful in Omboi we've actually phased out the use of plain JSON without types in v3 and I think we would probably be using type structs in various places in UDPA where we want some kind of structured expansion again this is very much the mechanics of how you automate versus what the API looks like there are a few of these things that start to land we also moved I think at some point this is a workload stuff there I'm not sure where we are with that but that was something that's probably not even necessary to the proxy to support other than as a consumer but it's got its own design dock not not not so important but anyway basically products have started to land here and Omboi depends on them in certain ways all these products have no Omboi specifics in there that's generally useful base types and things like that so this point I'm not so interested in speaking to basically it was just that we've started to think about one of the things that we need us to think about is how do we write an API specification which can generate docs and where the docs might be slightly different per project and I don't know if we have a good solution to that yet and maybe folks will I'm going to scroll to speak to what they would see that looking like but you know it's probably going to look something like we've got a base set of docs in a certain format for example Sphinx RST or something like that or that or some sort of doc generator which could be targeted to any format that you choose and that somehow needs to merge some amount of status like you know does this does my proxy support this or do I ever need proxy specific comments on this API with the base documentation and that might maybe we would have anchor points and you would be able to supply a manifest which you know maps each of these anchor points to you know the structured information for your proxy and you just provide this manifest file that the docs build or something like that I think that's the the way that I think ideally that would scale but we certainly don't want to put in sort of things like this if which isn't information yet in the docs that are landing the UDP repository there's obviously that's way too proxy specific. Has anyone had an experience with this kind of documentation building scenario across APIs or is there any thoughts there? I can ask some of my team members I think I think I know we did some things like that and definitely some kind of overlay approach would be useful perhaps to end up as as a special like notes or specifically formatted notes anchored to available anchors or even figure names or heading handing names or something like that. If I if I have any useful information I'll share back. Oh okay thanks a lot yeah I mean I'd love to hear it there is you know existing you know tools or languages that people prefer here and that you know well what works well in practice I think yeah the general structure of using anchors I think is probably where we would head towards but I I rather avoid you know not invented here and cooking something up from scratch which is that would probably go otherwise yeah cool okay so yeah I mean we have we have generated we have a developer generating or a community member generating like html docs out of basic eski documentation so that's like the worst possible example of trying to enrich documentation so like if we can if we were able to accomplish that I think we can sort of hopefully find a more maintainable and of the shelf solution to enrich an existing document structure with additional layers of info. Yeah that makes sense I mean for reference Envoy uses Sphinx which is you know some sort of Python based documentation system which a lot of projects use and it's pretty nice it's like I don't know if it's the best one out there but it's certainly it's got some nice features like for example you create internal links and references and those links consistency checking which is like really good for like scalable documentation and it's a lot of stuff. Yeah I have experience with Sphinx and it's it's a good thing yeah for sure. Okay we'll see if we can if I can get some information there I'll chime in on the mailing list. Oh thank you. Okay so you do pay next steps so there's a lot of I think the developer is pretty clear that we need to actually start doing some things in the repository which will be useful and I think that the probably the least controversial thing to start with would be the transfer protocol which itself is very controversial but it's at least opinionated about you know proxy specifics. The transfer protocol is more about delivering objects over the wire versioned objects and being able to deal with dependencies and being able to deal with you know acknowledgments and errors and all this kind of stuff and a lot of this is largely going to be incremental from XDS we already have you know the UDPATP specification which is it's so implied but I think the foundations are clear enough that we could start to build something within the repository and then actually evolve that specification as an as an as an actual code artifact but not as a low level C++ code artifact because that would probably be kind of a lot of boilerplate and really verbose but the idea is probably to put together some sort of a sexual specification which looks like the protos and something to actually drive those protos on the wire and pretend to be like you know a client in a server and so I was thinking of no we will potentially just start by writing this as a bunch of Python scripts and then that would become the reference implementation and from there we could build compliance tests essentially using these and have something which you know anyone who wants to claim the UDPAT compatibility with could implement and we could then think about how that one looking in terms of getting envoys for example to speak this and speak to the rest of the to do the reference implementation and there's probably already some choices there like for example like what would your specification or your reference model look like with the Python should be written in Go should it be written in Rust and there's all kinds of questions there which it would be good to probably hear from folks on because all things being equal I'll probably just write this in Python because I know Python extremely well and it's high level it lets you write a lot of express a lot of things without actually writing a lot of boilerplates and so on so yeah any thoughts on that? Have my thumbs up for the choice of Python as something readable and sort of close enough to a pseudo language description that other implementations can rely on for like almost like documentation itself. Okay that sounds good. Kostin asks well chat multiple languages to keep us honest that's an interesting idea. Kostin are you able to speak voice or not sure but the main issue here is I would like to just keep this as you know I did use one canonical reference here which folks can be taking as a specification so just write it twice and like once in Python it wants to go that seems kind of like a lot of work. I feel we definitely do want go implementation but there's a difference between like a product implementation I think that which we wouldn't actually use I'd say is your management server and something that's just there to say hey this is a modern culture look like I'm actually very interested in hearing from Mark on this because Mark has looked at the XDS protocol today and he's been a lot of very careful they're picking some they're picking but also just a very careful analysis and auditing of how it looks and where the gaps might be what are you the goal of this exercise is to avoid this occurring with UDPA and the new transfer protocol we put together do you think that just writing a Python reference would be that thing that would take us to that place with some tests or is there something else that you're about to see? I think the compliance tests are probably going to be more important than the reference specification although obviously you need you know something to you know make sure the tests are doing what they're supposed to do but um yeah I mean I the reason I think the compliance tests are particularly important is I'm not so worried that like if we only do the initial implementation in one language we won't be able to do it in other languages I mean I think particularly if we choose Python like we know Envoy is going to be doing this in C++ we know that gRPC is going to be doing it in at least C++ Java and go so I mean if there are problems that somehow don't let other languages implement the spec which is hard for me to imagine I you know I'm pretty sure we'll be able to identify those fairly quickly I'm more worried about just making sure that the spec is rigorous enough that we don't have subtle differences in behaviors between implementations and I think the compliance tests are going to be a lot more helpful with that than the the reference supposed to be the reference implementation cool yeah I'm I'm not going to implement that and yeah hopefully we can you know get folks involved you know she is looking at these tests as they come up about and the and that shows back and we can get we get it with circle box and like initial thoughts are probably too wide is what a reasonable time frame is to get to these basics of the type of protocol out there little artifacts and get to convergence agreement I mean we can meet back you know towards the end of Q1 and say you know are we in a place where we want to call this the UDPA tranquil protocol or not and you know most of the fun hopefully most of the discussion going forward can be other than being done on a google doc can be done as PR reviews and we can you know if we can sell it sitting with at least controversial speech in the UDPA protocol spec get work and write test written and then you know propose a concrete code review through code review extension like you know we need this for federation we need this for on the land support and we can then go work through those discussions in a very focused way so that's the goal there so I'm I'm sort of unless there's more boundless and more folks who are able to contribute towards UDPA in Q1 and maybe there are no it'll be interesting hearing from folks on this I feel we'll mostly be focused on transport protocol then and then Q2 and Q3 would be about data models using all the infrastructure that's been built in Q1 and Q4 would largely be at the where we sort of take this stuff into to a place where we can actually say this is you know intercept the onboard V4 APIs, TRPC we might adopt this other proxies might be adopting this and so on does that sound like a reasonable roadmap is there anyone who's you know interested in contributing boundless in Q1 or Q2 it looks like you're sort of getting an idea from folks that's okay so for us we're a little bit tight on bandwidth during Q1 I'll see if I can at least contribute with someone someone like war stories about for example the documentation enrichment and things like this we are our primary interest for sure lies in the Q2 scope of data models and making sure that we either have like sensible and extensible vendor specific extensions or at least generalized enough basic data structures and processes that allow us to expose as much of our a little bit of functionality as our users expect from us to have so yeah yeah that that makes sense and yeah I think that that's one word definitely really valued the insights from other folks because we just don't necessarily have the visibility into how other proxies work okay yeah I mean I feel sequencing this way makes sense because we do need to build some of the basic infrastructure for just doing tests and you know example clients and servers and transfer protocol is kind of needed before we can really you know do a lot so which is like why are realistic so that's kind of why I went for transfer protocol I also feel like transfer protocol is going to be controversial because for sure if it's a proxies I mean there's a lot of discussion about do we need this this is very complicated to implement yeah yeah we've already had some of those discussions but I think those things concerned alone and they're there that that things we can get into within the time frame of Q1. On this topic I mentioned this before but one way to avoid some of the controversy is to maybe start with subset of the existing XDS protocol which has most of the elements we need anyway I mean it's a generic bi-directional message passing protocol with any and typed and everything else and then iterate on it and make changes as necessary I mean if there is strong justification but we can just put straps the process and have some initial versions that is existing XDS minus all the data model and then fix whatever is necessary so we don't start in avoidant bike shed until we yeah I mean yeah I mean a complete guarantee that constant I mean it'll look kind of like the existing XDS and I'm not going to guarantee it's wire compatible but I think that's where what I was thinking of is we would start with like it's simple enough that it looks very much like the existing XDS because a few slight differences that were there I mean was there a reason that you might want to wire compatibility with existing XDS? I don't care about wire compatibility I care about avoiding the bike shedding and you know making arbitrary changes just because we can I mean if we start with the existing protocol assays I don't care if we change some protocol number we add or whatever but at least we have a basis that is solid and we know we understand and has multiple implementations I think I think what I mean I think the the UDP UDP draft that that Harvey put together is is actually a really good place to start I mean it is very similar to XDS but it I mean I really cannot tell you the number of surprising and and difficult to deal with education that we've run into in gRPC trying to implement XDS there are things that just semantically do not make sense in the current protocol as is so I do think that we shouldn't do anything dramatically different I think custom I think you're right about that but I don't think just starting from the current protocol as is a good thing I think we really need a we need to start sort of fresh which is what the UDP draft does with the same concepts but with a clean thing with well-established semantics that doesn't have any of the backward compatibility problems that that you know tie us to these ridiculous behaviors that are grandfathered indexed yeah yeah yeah I don't disagree with you on is that many are off ages and it'll be wonderful to identify them and fix them but there is no guarantee that if you start fresh you don't repeat the mistakes I mean or you don't end up with something that's us because you know usually iterating it's you know you fix what is broken you keep what is good instead of I don't know that's my experience of course I think like the right way I think about this maybe to think of this is like from a historical perspective so you know XDS when it arose it arose you know we had an idea that it might be more generic than just an onboard but in reality it was built in the onboard code base the only test that existed were the onboard tests the only reference client was onboard and its internal implementation and then led to a number of decisions creep in when you do that so the idea with udpa is we get very clear up front the test for all these crazy corner conditions and I'm hoping we can really help us that's what some of these might be as we go because we want to make sure that you know for example like all that are on our various error cases and situations where there's singletons or there's like you know a failed lazy on the manual load or things like that that could be we have like you know carefully designed things which didn't just essentially evolve into the protocol which is where we are with XDS today and I mean there's been some pretty pretty stuff like you know how we deal with like names and aliases which is still in the ongoing topic of discussion and so this to be fair we can only really have a good discussion about after we had all the experiences of not having it done right the first time and so we're going to ideally just build them out in udpa tp like we may really rethink what aliases look like in udpa tp to make sure that that it's very clear why they're there and what what what purpose they sell but maybe in the first drafts in the first PRA land let's say there will be no aliases well strip the aliases out and then we'll full design discussion around adding aliases an alternative would be to to start with an existing protocols that is you know for messaging like like the NAT streaming or some other protocols that exist and modify to our fits but again if you think that it will we can avoid the bike shedding and and will not repeat mistakes by by trying to be genetic and have second second system syndrome uh maybe see how it evolves in in in the last next month or so and and if it ends up with something stable and we don't go down the rabbit hole yeah and the thing you see in mining is ready for you know order 100 or so management servers out there many artists in house management servers written for xds and so we don't want to go too far away from the existing model like I don't want to move off to some generic like the message bus architecture or something or message streaming system which has nothing to do with you know the the sort of transfer protocol we have today but that's a big rewrite for a lot of folks I feel and it's also just yeah like it's a big conceptual step at the same time things we should just play in weird that we've done there in xds the wrong way we've done better in other messaging systems we definitely should build for our uh taking to account like the other folks do ever handling better or um you know versioning better and uh and we've missed something that we should just build that into the protocol yeah yeah cool um so that's all I had on the agenda uh part of the part part of this video just to make the play that udpa is still alive I I as a tube con and uh the folks were like yeah I heard about the udpa thing but haven't heard anything for a while the the the down to has been we've been very very busy with the v3 apis and now that that work is basically uh tapering off uh udpa is the next focus from an api perspective from the at least from the old boys folks and uh and for myself cool that's okay well thanks all for joining um I'm going to schedule another meeting in let's say probably mid quarter and uh single game then so the next meeting will be in another time right yes this is the us ton zone uh is it Europe next I think Europe's next um I don't know if that helps you out much uh no no I'm not asking for me because the developer people is asking to join the meeting and this is quite not apex apax friendly yeah yeah I I realized that um helps you I can do to make it friendly to europe and apac and I know there is uh someone in Australia yeah I think we agree don't do the rotating so I think that's fine very fine for next time I think they are very interested in like taking the more like data model stuff so okay then get more people involved um yeah that's um I'm going to be joining us next time this time he was traveling um if he's been on vacations cool okay cool then the hurry just quick uh meta roadmap question so at which point do we expect um roughly other proxies to start adopting dpa I assume that's going to be after we start defining data model which is actually going to be some of practical use for those proxies yeah I mean I think there's there's there's multiple reasons proxies will adopt u dpa um some more adopters are almost just going to adopt it because it's part of our api roadmap grpc is likely to do the same because he wants to it's essentially tacking to the same uh uh api roadmap so that's already two we're talking about um uh there are other proxy a lot of folks for example Citrix has written an xds adapter which maps xds to their own internal um what uh consideration you know protocol or language and that's done because they want to be istio compatible and they just call their istio adapter and then they're um they're basically they've adopted xds and so once they need to do that for istio compatibility I feel that will cause other folks to then pick up a u dpa or part the parts of u dpa which are necessary for istio compatibility now I don't know where the a j proxy folks are I know your folks are they're definitely uh checking on in many of these discussions I don't know if you're where you are with your api roadmap but um I'll let them speak to that all right cool all right thank you thanks Harry okay thank you for the meeting guys thanks bye