 This is why we have friends. Exactly. So let's see, it's five, six past the hour, so I think we can get started. Do you want to drive or should I drive? Since I can't even find the mute button effectively, I think probably it's best if you drive. All right. Okay, so in that scenario, let's go ahead and get the meeting notes up. Is anyone able to share? Great, thanks. Okay, while we're getting set up, so first is a reminder, please add yourself to the attendees list. Second, please remember that this is a recorded call. Third, if there are any topics anyone would like to bring up or any documents or anything that is not on the agenda, then please bring it up. I was not hearing. I'm sorry, what was that? It was Yvonne, I just said hi because my audio was not. Okay, so the first action item that we had was with, okay, so in terms of the release notes, let's go ahead and get started there. So overall, I personally am pretty happy with how the initial set of releases, how the initial set of release notes were looking. Let's see, that's against the changelog. Yeah. So let me see if I can find the... In Google, maybe this is here. Yeah, that's the one. So there's still a little bit of work that needs to be done before we can call this one. I'm kind of complete. So in terms of tasks, I wanna go over some of the details if possible. So in the getting 010, are those the three areas? We have Helm, Docker images and signed gets tags linked on GitHub. Are those the three things that we want to show off? Do we wanna add anything or remove anything? I mean, I guess that the release notes also are kind of the release also and maybe some form of documentation. I mean, is it going to be part of the git tag? I mean, how do we... I mean, I think that at least the release is that is still does, okay, let's get this too. So they essentially have archive of the Helm, a couple of readme's, maybe some, for example, configurations, things like that. And then everything is downloaded off of Docker. Okay, so let's see, in terms of... Yeah, so that is a possibility. We could do a tar ball with the Helm charts in it and that would definitely be useful. So, yeah, that's... I think that's a good idea to add. So, for images, so the plan is the plan to... Should we keep the plan to publish everything to Docker or do we wanna do a simultaneous release to the new GitHub repos or who we wanna just leave the GitHub repos alone for now and just stick entirely with the Docker, which I think is probably more well-known at this point. Yeah, I tend to like putting things in Docker, but we do have to think a little bit about where we stand for our Docker repos, because right now we're company our CI and the same thing as everything else. And so we may wanna consider essentially reorganizing in such a way that basically we have a separate repo for the CI stuff. Yeah, I think the suggestion last time was we have three. One is the release repo, where we basically put things that are actual released Docker containers properly tagged. The CI repo, where we do all the farming off of images for CI and then sort of what you might call the Prax master repo, which is the repo where we go put the things that we are actually building off of the master branch that have gotten past CI and been merged. Yeah, and something that we could consider in this scenario is, so I don't know if you saw a GitHub announcement from about a week and a half, two weeks ago. So they are now shipping, they are now attaching to every Git repo a Docker repo. So you can actually do a Docker push and pull to a GitHub based repo that tracks it. Yeah, I would love to do that. I have actually put us in for the beta for that, for network service mesh. I see, so it's not properly released status is what I'm hearing. Yeah, no, as soon as I saw it, I went and signed up network service mesh for the beta. And we're also by the way in the queue for the beta for GitHub actions, which would be kind of cool. But the best of my knowledge, we are not actually in either of those betas yet. Actually the GitHub actions is a really great example because that came out a long time ago and they're still not out. So I guess we should not expect to see this for a considerable period of time. It's hard to say the thing is like, I had originally, because I wasn't thinking about it when GitHub actions came up I signed up for my personal repo for my personal organization and that came through. And then I realized, oh wait, I'd actually need this for the network service mesh or again went to sign this up for the beta list then. So some people are getting them. It's just hasn't come through for us yet. Yeah. Okay. So I think that from an action item then, and I know this is a documents meeting but we can hash some of this out here, is perhaps what we do then is we have the main production repo which could be like network service mesh. And then we separate out the CI into something that's less noisy. And that way when we ship, we only ever ship to a release branch we're not polluting the other branch. Another strategy that I've seen as well is for people to create a development branch. So they'll have like colon DEVEL and they would then drive the master branch to that. But I think one of the things that gives me a little pause on that is that I think our approach is a little bit more complex than the standard application where you could build local images, see it work and then discard the images and then move it forward because there's no distributed aspect to it. So I guess the question is in the scenario is that something that we would want to do in before this release and to make sure that the split is there and properly working, it is a bit close to the release which worries me a little bit but at the same time, having something that's clean that we can put out there I think would be of high value. I would postpone it for either. I mean, we have enough things for this one. We can never make it perfect, I don't mean. No, that is true. Okay, well, let's go ahead and postpone that and what we'll do is let's just make sure that when you do the default pull that we're not overriding the default master and master should always point to the latest to the latest release branch. So let's just double check the infrastructure to make sure that that assertion is true and then beyond that, we can punt any splitting or any other things like that up to a future time. And then what we'll do is we'll describe people that they can either grab off of for the Docker images, they can pull off of either Docker master to get the latest release or they can do Docker zero. So here's a question. So we have, when we do a pull, like so as you're pulling go, you can pull like Golang master and I always pulls the latest. You put Golang colon one and that pulls the latest one dot X branch, which right now there's only one major release. But when 2.0 comes out, then that means that people will not automatically upgrade from 1.0 to 2.0. Do we want to adopt something similar in this and do the release where we have like, get Docker, sorry, get network service mesh zero and put a tag for zero one and a tag for zero one zero and I'll point it to the same image for this point and we can upgrade over time, the images appropriately. Yeah, this sounds quite reasonable to me. So yeah, I mean, it sounds quite reasonable to me that's the way we can go about it has the advantages of not proliferating repose a lot. Yeah, and we focus on tags, there's a new release and then we rely on the tags to allow people to select they want. So if someone wants an exact version zero one zero, I don't want to move up to zero one one until like fully validated, everything works, then things are okay. And when we start to hit the full releases with NSM where we have our first 1.0 release with semantic versioning and then this will work beautifully, I think. It'll allow people to work out how much risk do they want to take or are they willing to accept in order to get later features or bug fixes. Okay, and then we're signed to get tags. Do we have a signature yet? We should probably create like a PGP key and then we can actually do a signed release so that people could verify against and publish the public key on perhaps maybe on Keybase. That sounds good. The one question I do want to ask though is how do we go about securing the signing key? Because the one concern I would have is how do we go about securing the signing key for the images? So in terms of signing, perhaps that could be a manual process to start off with until we're comfortable or until we find a alternative that we can automate. Because right now, my main problem is that we unlock the key and that key is sitting on a shared repository or on a shared system and I'm a little uncomfortable with when the image goes away, can we have a guarantee that it's always gonna be cleaned properly or are there gonna be other site channels, some processor attack or something similar where they managed to dump the region of memory that PGP likes to store things in. So I'm a little concerned on that side. Yeah, I have similar sorts of concerns on that side. One thing I think that we can probably do one thing that this sort of brings to light immediately is we should definitely do some key signing like you kind of you next week. Because that way we've sort of at least got each other on the same web of trust. The good news is I, for reasons historical, I'm a trust anchor for Linux Foundation. So I'm a very highly trusted key. And so you're going to sort of get the web of trust going within the NSM community as well. I think that'd be a fantastic idea and what I think we can do is we can sign and we can print out on paper or something similar, a key that we can then seal away and we can work out how do we want to properly secure it so that it's not lost or stolen easily. Well, I mean, I guess part of what I'm saying is at this point, if we're gonna use PGP rather than having an SMT, we just have one of us sign the release. We've got some attestation from a public key. I presume how many of you folks on here actually have public keys currently? I got a public key, but it's not a common thing. People... Yeah, no, it's not a super common thing actually. And it might be sort of worth thinking through how we want to do it because public keys are kind of inconvenient to work with. But I'm all for having some way for people to be able to validate and verify an image. Yeah, and what I was thinking of is if we can publish the key in a couple of ways. So of course, we would publish them to the standard PGP key servers. We can publish it into network service mesh website itself which is gated with Git and secured with HTTPS and publish the fingerprints and so on of the image. The third place that we could also stick something is, again, the key base is gaining a lot of popularity and they make the tooling relatively easy to for people to set up their keys so that they can then verify images and so on. So of course it begs the question is to like, do you trust a third party entity to set up your infrastructure for this type of signing? But if it's for the purpose of verification of the image, I think that that may be an acceptable risk. We just have to see like, do we have to give key base the private key or can we publish our own custom key? And I think if we can do a custom one, it should be okay. Yeah, so a lot of things to look into definitely. Cool. So I'll take a look into some of that stuff and I think we'll work out a strategy for and I think what you described is perfectly reasonable. So we sign each other and any one of us, like we either select somebody as the signer or perhaps we can do something where any one of us can sign and verify the keys. So we can work out a strategy on that, but let's work out what our primitives are first. Cool. Okay, so in terms of demos, so has anyone gone over the recently over each of the demos and made sure that the demos work? So we actually- I am, yeah. So we do have the testing that you put in July for the make version of the demos that's currently running to cover some aspects of them. I did notice that we had a failure in there. We have some intermittent failures that are happening. So I'm trying to pull the other something that does better logging. So we can figure out what's going on there. And then I think it would probably also be good. I think the way we want to direct people or in demos generally is probably via helm rather than via make. Does that make sense? Yeah, that makes sense. And so the question I have in this scenario then is the helm infrastructure mature enough now where we could reasonably do that for zero one zero? Yeah, the only thing, I've used it successfully to demo in the past. The only thing that I would actually really prefer to see is being able to break out the clients from the services as separate charts. So you can say, look, you're a home install network service mesh. Great, now you've got network service mesh enabled. Oh, you can monitoring and this helm install NSM monitoring. Great, now we can see monitoring. Oh, you'd like to go VPN available, home install VPN. Only you'd like somebody to consume it. Well, helm install some home chart for the client because that sort of flows naturally in terms of what day one days, zero day one day two looks like. Okay, that makes sense. So in that scenario, I suppose the only thing that we need to do in this is to, so it sounds like automation is there. Is that workflow in CI yet or is that something we should build out? So the helm stuff is not actually in CI yet. The mixed stuff is being tested, which is important. The helm stuff is not. Nikolai, do you have any thoughts? I know you're the one who did the make testing, which is crucially important. We're breaking up really badly, Nikolai. I can't really understand you. Nikolai, you might help with you. Did you type for a short while because you're breaking up quite significantly? Like we're getting every other vowel. Okay, I think we officially lost him. Yeah, I think you may be trying to just connect and go the direction and figure out his audio stuff. Okay, so. I'm absolutely with you though about testing, figuring out what the demo is, getting it cleanly documented and testing the type putting it in the CI. One of the things that I've noted is I've had a couple of people internally been asking to go and keep the tires on the code and sort of we have all the pieces but they're in various places. Like we have the quick start but it's oriented towards the make files. We've got the home docs but they don't tell you anything about how to get a cluster. We've got some stuff about how to do the checks with make check in the quick start but we don't have that stuff related to the home charts. We have, I don't think we have any documentation about using Skydive to go and visualize what's happening. So there is sort of the traditional scatter gather problem in documentation. You write excellent boxes, you're doing things and they don't all quite hang together as a unitary whole. Yeah, so I guess the part of what we need to do then is come up with a plan similar to how we drive the release itself. Perhaps what we need to do then is come up with a Google document or an issue or a spec that we can then track and say here are the documentation tasks that we need to do and that way we can track each major item and make sure that the progression gets done. So it could be something like make sure that how you spin up a cluster with Helm is properly documented. How do you, and just to get like an outline of what needs to be in there. Well, I mean, it turns out actually that the Helm docs are actually really amazingly well done. You did a good job on the Miliya. They even include things like the following is likely going to have to be done if you get an error like this because there's a degenerate case where you have to go run a few Q and control commands to set up permissions for things. So they're very well done as far as they go. They're just capturing the Helmi bits which are important to capture. But again, if you're sending someone to the here's where to go kick the tires. You end up jumping them from doc to doc to doc. I see. So the issue that we're having is we have good documentation. It's just we need to lay it out in a very and a very easy to follow way. The getting started guide is excellent. If you're going to start and you're going to use the make machinery. The guide is extremely well done. If you have a cluster and you just want to run a home chart and you're not worrying about how you're going to convince yourself that anything is working or see how it's actually going on. And I think this is actually right and proper for those guides as they stand. I think they're actually scoped correctly. If you do want to be able to point people to the oh, you just want to get home going. Here's what you do. But we don't really have sort of a demo walk for you. And that might be helpful. Okay, that makes sense. Okay, I'm back. I hope. And you sound great. Okay. So wireless range extenders actually help and do their job, right? Okay. So I wanted to talk a little bit about the example. So right now I'm working on replicating the demos that we have in the main repo, in the examples repo. And I already have two more demos and have examples there. So it will make it because I split kernel ICMP and PICMP into separate repos. This will make essentially five demos in the examples. So I know that this is a little bit of a kind of controversial topic that we have discussed about this back and forth. And maybe for the first release, we are just going to go with the main repo. That makes sense because that's what we have today. But maybe at some point we could just point the people to these examples repo and everything could be there. Yeah, I like that approach. I'd even make a suggestion and please know this is only half thought through. So don't take me too seriously. I'd even make a suggestion that we would be going more granular, right? So if, for example, we were to go and say, look, we have the VPN, imagine doing a repo for each network service endpoint that we could basically use because then imagine that the way the CI ends up running. So if I just go and I change the ICMP responder, then the commit goes into that repo, its CI goes and tests to make sure the ICMP responder NSC is still working. And if it's working, then we go. And so we don't have to test the broad swath of everything because that has nothing to do with the ICMP responder code, right? As long as it properly comes up and does its thing and a known to be working version of the network service mesh, then we're done. We don't have to go test the crazy auto healing stuff and all these other things. And that might significantly speed up the CI by basically breaking things down into smaller repo pieces, but. Well, I mean, then how you should have separate CI's for each of the examples. So this is actually a really common pattern when you have a little bit more control over your infrastructure. So I'll give you an example. The people at ThoughtWorks, which is headed by people like Martin Fowler created a build system which they unfortunately named Go. And, but one of the things that they did was they had the insight from experience that when you build a project that you have, that you do have dependencies on how the projects are laid out and you have a set of pipelines. So there is some similarity in CircleCI with pipelines. But part of the issue that ends up running is that when you trigger a build for, let's say a network service client and you don't change any of the APIs, then in that scenario, it's like, what do you have to rebuild? Do you have to be a rebuild NSM manager? No, you have to rebuild perhaps your demos and rerun those and so on. So you end up with this dependency management thing that occurs, so that when you trigger things that in order to make sure that they continue to work against, as you said, known versions. And in the long run, what ends up happening is you would then peg things like the NSM manager to a semantic version itself. So instead of having the umbrella semantically version, which you could still do, but each component gets their own semantic version with similar properties that you can then use to to verify that things work or don't work and control the artifacts as they change. So there may be some strategies that we can look at. I'm not suggesting we use their build system because I don't think it'll be a good fit for us, but there may be some strategies we can use to simplify and head towards this pattern as you described. Yeah, let me be super clear about the pattern I just described. Number one, it's a half-baked idea. Number two, everything you're talking about doing for the examples, repo-nigli, is actually a positive step forward. Whether we do or do not have to fight it is a good idea. Please don't slow down or stop at all because I had a harebrained idea. No, no, no. I can tell you. Well, I mean, sometimes it happens. Like, you know, sometimes, you know, somebody's like, oh, I guess there's this better way we're going to do it. And the answer is, I have no idea if this is a better way. No, for me, so what I see now with examples that essentially I have, for example, nightly builds, which verify every day that the examples that I have for building and running against whatever we have in master, or at least the published latest images, that's more clear because I don't build images. I just download whatever is out there. And I don't know. I mean, if you have to do this on five different repos, example by example, maybe, I don't know. Yeah, I know, there are trade-offs and you're absolutely right. It may prove to be a massive pain in the ass. I mean, this is always the trade-off. Like, great modularity brings both costs and benefits. And there is some point at which it stops making any any damn sense. And I just don't know where that is here. But either way, I think breaking down to examples is probably wide. I would be unsurprised if it turns out that we don't quite get there for the 0.1 release. But I think the work is still really important. All right, so in terms of actionable items for the release. So it sounds like the main thing that we need to do is ensure that the documentation that exists across the board and the documentation that Ilya put together for Helm gets adapted in a way that we can say, here's how you run a demo. And make sure that those are tested by people who have less or no experience with NSM in order to make sure that they're readable and understandable. I actually have a classic that I'm gonna use. When we did the initial open sourcing of VPP, there is, I went and got a director. Try the instructions. Yeah, would that right for in mind? I do, I do. He's gonna be deeply amused, but possibly not cooperative. But the thing was, when he could sit down and from the instructions cold in less than 30 minutes actually have things working, that's when I knew we had good directions. Cause he's a bright guy, but he probably hasn't touched code in 15 years. That sounds good to me. Okay, so let's see, let me make sure this gets added to the release notes. Oh, okay, yeah. Yeah, I added as a to-do item. So you can do a search for to-do in the release notes area. What I'm going to do is I'm gonna go in after the fact and start putting together a documents board, create a new board, and start outlining the various things that we need to do in there as in order to improve and get to the state where we want. So we'll turn these into GitHub issues effectively. So interestingly, this could probably live on the, this could probably live on the Network Service Smash site page because this is where I think most of the work's gonna be done so we could then match them to issues there. Let's see, in terms of the, so in terms of the document then, I think those are the main ones that I wanted to cover in this one. So I guess one last thing is on unknown issues. What do we want to do on known issues? Do we just want to leave the generic thing that we have at the moment or do we want to call out very specific things? Yeah, I mean, we could just say it's awful, please. Don't consider it in production if that's the motivation. The way I usually put it to people because it's a little fettler is to say this is our 0.1 release, which would be treated exactly like a 0.1 release. Yeah, and I'm happy to leave it as a generic thing right now because I think enumerating everything would just be a waste of time unless there's something really big that we need to call out. Don't use VPP in this way because a kernel bug will cause your kernel to panic. And even then, I'd still be skeptical of whether we should stick that in or not. So how about we leave it as it is at the moment and then once we actually get to a proper 1.0 release and then what we can do is we can create a separate known issues page document because that's also, I think, should not just be like a static page. It should be something that over time as major issues are discovered, we can then populate that page on the website. So here are the known issues for the 1.0.1 release and that way people can then track and make sure that they see what things are going on and so on. Does that sound reasonable? Cool, I don't have anything else on the release notes at this point. I think it's just a bunch of task items and then once we're closer to doing the release we can go over and make sure that everything is as we want. So as you know, nothing's set in stone until we sign the commit. So let's see, it's back to the meeting notes and let's see who's on right now. So it doesn't look like we have any of the Volk people so I can't ask them any questions regarding to the videos or anything like that. Nikolai, I'm gonna spend more time on our presentation. I've been slammed with my time schedule but things are opening up now. Now that I had a product release that I had to push through earlier and that's done. So... If you are available, we can do follow up after like this call. Yeah, the only problem I'm gonna run into is I'm supposed to be checked out of the room in 15 minutes, so I'm not gonna have too many more. So, but I have a persona in mind in terms of how I want to try to approach it and that's all pitched out to you. I'll pitch that to you and I'll put together some slides at Wernicke style and see if it meshes well with how you're thinking of things. How you're thinking of things. Yeah, slides already, I mean. Yep, so I mean, I also like for the intro stuff we still need to get some slides together as well and those are probably gonna come in a little bit hot. One of the things I would love to do specifically for the intro talk is we have a brand new color palette, we have a bunch of new graphics to work with. I may tweak some of the iconography, but so. Oh, hey Ed, if you wanna hear something a bit comical I got another email from the Fido Summit saying that I needed to register. Sorry. And I went in to register, I paid the 50 bucks for the other one and it turns out that it's a radio button so I can't register for both. So I let them know. But just a heads up, they are charging us for and they are charging the speakers to speak, so. Yeah, it's just, yeah, it's grumble grumble. Yeah. Okay, so the only other thing that's really bugging me at this point is the what is NSM to replace on the site? Because at this point my inclination is just to delete that article. And then we can, Jeffrey said he was coming up with a first draft and he may have already done so because he's pretty on the ball like that. And so perhaps for the what is new we can, if we get a new version of that then we can stick it up but I'm not comfortable with what's up there anymore. The thing was written over a year ago and has really old concepts that have moved on. Yeah, I mean the other thing I wanna make sure we do is that we sort of fully express the breadth of what we're doing and where we're going because while we absolutely are gonna be taking care of the NFE guys, I think we actually have even more going on on enterprise than we do on NFE. I agree entirely with that. And so, yeah, I mean literally every enterprise networking guy I talked to who is coping with the cloud transformation stuff, they like their eyes light up immediately when this thing becomes a plex because they only start talking about the use cases they need it for. Yeah. And I think now that we have the envoy as an example let's see this could speed up. Oh, I'm super happy about that by the way that makes me so happy. Yeah, the envoy one is easy. It's also makes it very easy for me to deflect the Istio question. Still like why, well not really deflect but give a good answer because one of the questions that I get asked almost every time that I speak with someone is why don't we land this thing in Istio or why not contribute it to Istio? And I usually have two responses to that. But the one I've been using most recently is that well, we don't focus on Istio because it's an L4, L7 and L4, L7 you don't have L2, L3 consume L4, 7 you have the opposite L4, L7 consume L2, L3. And we are providing envoy. We have an integration with envoy which acts as a network service. And that envoy could be controlled, potentially controlled by Istio if the Istio people choose to do so. Yeah, which actually also reminds me do you guys feel good enough about that example that you wanna sit down with Matt Klein? Oh, we know Matt's not gonna be in KubeCon Europe. That's right. We could arrange a call after KubeCon Europe and sit down and show him what we got and or we could potentially show what the envoy community got when we get comfortable. Yeah, I think that's a good idea. And we can also show him a concept of potentially integrating it in with the Kubernetes networking in as well as opposed to just a side thing if that's a direction we want to target. But I think even just showing off the envoy as we have at the moment, I think would be fantastic because people can ask for whatever service that they want with an envoy that's been created by someone else as or by the operator and get much better control over how these things work. Yep. Yep. Right now, running envoy as a side car gives the user control over, gives the pod or whoever has access to manipulate that pod control over envoy as opposed to the operator. And so that's a pretty big shift that we're proposing. And it's one that I think will be very well received. So I'll add that. That's it. I don't have anything else. Is there anything else that people wanna talk about? Cool. So in that scenario, let's yield back 10 minutes of our time and I am looking forward to seeing each of you in Barcelona. Yep. Probably most of us. Yeah. I'm looking forward to seeing you sooner, Ed. A tiny bit, although it'll be a little bit of the afternoon before I say I'm gonna go in, so. Not fair enough. Fair enough. All right, talk to you later. Okay. Bye. Bye.