 Hello and welcome everybody to the OKD working group meeting. I'm going to sub in and chair this one. If you have a chance, please enter your details in the HackMD file. And Timothy is noting, HackMD is laggy for me. Maybe we should archive this one and create a new empty one. How about if we do that after today's meeting for the next meeting? All right, Timothy. Now that I've got the agenda in this one. Yeah, it could be HackMD, could be the universe. So today we are supposed to have a talk on micro shift, but the guest speaker has not shown up yet. So I think what we're going to do is we're going to motor through the agenda and see if Sally O'Malley or one of the members of the micro shift team do show up. And while you're talking, I will. Hello, Neil. I will motor through Slack and see if I can find her online and make sure she has the details for getting in. Because Vadim is the one who invited her and she's there. So today we are going to just power through. I'm going to share my screen because it's easier for me. Difference between Jamie and myself when we chair. I would like to put the agenda up on the screen. Now I can see it and you get to see the rest of this. So Vadim is not here. If you, Christian is, you want to give a quick update on the OKD release on behalf of Vadim? Christian? Yeah. So Vadim told me that there isn't much news on 4.10. I think he didn't do a new release for 4.9 this weekend. I think due to the circumstances in Eastern Europe. So, yeah, no big updates from engineering side here. Yeah. Where is Vadim based out of? He's not in Ukraine. Is he? So he's based out of Brno, but he is from Belarus. Oh, well, so that makes it complicated. Everything is complicated over there right now. That's just pretty clear on that. So, yeah, definitely devastating what's happening there. Yeah, we have to be cognizant of that for all of our colleagues as we have offices and lots of Eastern European countries. So, arts go out to that part of the world. All right. So, and anyone else have anything to add on the OKD release? We've been playing with it or anything you want to add in? Otherwise, we'll move over to the F-Cos update. Timothy is typing furiously. May have to take yourself off mute, Timothy. Yeah, yeah, I'm, yes, I'm typing the agenda quickly in the height of D furiously, as you say. So, yes, so not much my side as compared to last two weeks ago. Essentially, you have a bunch of things in progress that will have come with the rebase to Fedora 36. So Fedora 36 is coming, the rebase for Fedora is next is coming along the beta, which should happen sometime mid-March. I'm correct. If I find the schedule again. Yes, the beta release is approximately mid-March. That's about it. So at this point, for example, we'll start making new Fedora next. This is based on the Fedora 36 beta, which will come with the Podman V4 change and the AP tables ineffable change by default. Those two major changes that we're tracking, we've added Docs page here. So the last link in the list of mail is a new page, new fresh new page that we've added to Fedora Chorus Docs, where you can track major changes that we have planned incoming in Fedora Chorus. So right now, those are the two main ones, but hopefully we'll have more. And there's also links and descriptions and how to test them on Fedora Chorus notes right now and when to see them happening and everything like that, the timelines. So yeah, and apart from that, we have all the Fedora 36 changes that we're tracking regularly. And I don't have anything as major right now. Thanks. Any questions for Timothy from the group? If not, we're going to zoom right along here. Not really a question, but I wonder, I think we're still on a Fedora 34 base in OKD right now. Is that correct? And OCP 4.10 is due this month as well. So we might, and I'm not sure if that's gonna be a problem, we might be skipping Fedora 35 then. Maybe we'll need to cut a release that is based on Fedora 35 before that can happen. But yeah, I'll definitely check back on that as well. And maybe Timothy, you know anything that might be causing problems? So I don't know what's the current status of the OKD machine noise content which release it is based on. I have to check that again. But for sure, that's one thing I wouldn't recommend trying is skipping a release. Like that's definitely something we don't test that much. Rocker is side of things because we have barrier releases for each release. So usually we don't skip releases. Like everybody goes through the flow where they don't skip major releases at least. Although we don't really think of the barrier releases. Those might happen within one Fedora major version as well, right? I don't think we've really thought of regarding those barrier releases. We just read because it's a recompose. It's not, we're not really using Fedora CoroS other than the, for the boot images. At least right now there definitely is a plan and there has been some movement on that too to not rebuild Fedora CoroS. And I think with the CoroS layering effort that's going on right now, that'll be much, much easier by then. But in general, we haven't really been looking at what is a barrier release and what do we have to, yeah, what do we have to look out for when kind of moving from one version to the next. So yeah. I'll definitely check back with Vadim and see. Might just make sense to kind of make one last 4.9 release on moving it to Fedora 35 and then when moving to OKD 4.10 also moving to Fedora 36. We'll see about that. We'll definitely come back here with some info on that. Can I say something to that? I hate, and I say that with that word on purpose, changing F-Cos versions within an OKD release. That breaks so many things and we've had so many issues with that. If we're gonna do 35, can we wait till 4.10? Please. I don't see any. That actually sounds like a, something screwed up on the other side of it because like the F-Cos people make great, take great pains to make sure that there isn't that much of an interface change between Fedora rebases. So look at 4.6, 4.7. When we change F-Cos from versions 32 to 32 to 34, it broke OKD bad. I don't see any reason why we should be updating F-Cos within an OKD release. Let me turn this question around. Is there a reason that OKD shouldn't validate that the upgrade procedure, that upgrading F-Cos releases should, is there any reason that they shouldn't be validating that that works? It's not OKD that break. It's all the stuff underneath it that. But it's all one piece. That's like the fundamental thing is that when you're, when OpenShift declares that it's going to do an upgrade, like, and I'm using the term OpenShift because I'm talking about both sides, both, you know, R-Cos and OKD, R-OCP and OKD, when OpenShift declares that it upgrades, it goes through and actually checks for updates for the operating system it runs on, as well as the software it's doing, and it attempts to do both together. And if there is event where either one fails, it's supposed to stop the whole thing. What you're telling me is that it is not stopping and it is still breaking. So what should be happening is that either A, there's a mix missing in the gap, in the test gap, there's a test gap somewhere, or B, something MCO is not, MCD is not doing something correctly to ensure that its settings persist as it upgrades the system. So, let me give you an example. So, FCOs changed what was a very, a very, very, very small setting, back in 34, 35, whenever it was. And it broke the fact of being able to do, they put a dot in domains, you know, for your Etsy host. That completely broke a whole bunch of stuff. That's not a test case with an OKD. It never was, and it probably will never will be. What I'm saying is that from my experience, and I've been in this now for quite a long time, so I think I have a right to say this, especially using OKD in production, that when we do an upgrade and we change FCOs version within the release of OKD, we are at big risk and we have seen it over and over again. My preference would be to either have a choice or not to do it and do an FCOs change when we do a new release of OKD. So I'm going to pause for a minute because I know you're on your cell phone and you can't see people raising their hands. So Christian wanted to say something to that, so let me just pause you. Yeah, I know this isn't on the agenda, but this is just one of those things that's giving me a hard turn. That's okay, I think it's a good topic. So go ahead, Christian. Yeah, I'd say we really have to think about OKD, not really in terms of OpenShift versus Fedora-Coroas life cycles because they aren't aligned. So we will always run into this issue and we do test the upgrades from one OKD release to the other. So it is one bundle that is upgraded and tested as a whole and it shouldn't really matter what version number it has. If the upgrade works, that would be our signal to say, this works, we can release it as is. And if we now move to 4.10 and then with that also upgrade to Fedora-35, by then Fedora-36 is out. And what are we going to do then? Are we going to wait until OKD 4.11 is out to move to Fedora-36? We have an OpenShift release, both OCP and OKD, roughly every three to four months. And we have a Fedora release every six months. So those life cycles just aren't aligned and we can't really say we move them along together and only upgrade one when we also upgrade the other. So at one point we'll have to say now we're moving within, let's say, OpenShift 4.10, we move from Fedora-35 packages to Fedora-36 packages without actually adding a minor version to the OKD version number. How long is an OKD release supported? I thought it was just until the next one. Well, yeah, it's a rolling release. We don't really do releases of the last minor version once we have the next out. So once we release 4.10, like only in emergency cases, but really it's a rolling release and there isn't any parallel streams like you'd have on the product side. So if Fedora is supported for a year after release, then I understand that the two don't release at the same time. But whenever a minor version, if we walk whatever F-cost is out at the time of an OKD minor release, that F-cost version should be supported and updated for the entire lifecycle of that minor of an OKD release. Am I missing something? Unfortunately, Fedora-CoreOS has opted out of the full Fedora lifecycle. Oh, OK. So that means that when they do the rebase, they stop supporting the old stream entirely. OK, now I understand. Sorry. All I can tell you is that from what I've seen in the last year, we have had many, and I've spent many hours debugging changes between F-costs. When there's nothing wrong with OKD, it's the underlying OS that has caused issues in the stability of OKD. It's not OKD, it's F-costs. That really sounds like MCD is not doing the right thing, because it should be forcing the settings and the environment to be correct on upgrade. It really mostly is things like a new network manager version or a system D version that has some change in API or some new feature or something that isn't properly supported, changes something in the output or something. That is what most of the time happens, because the OKD code, the cluster site, that is what we also release as the OCP product, which obviously is very, very well tested. And then bundling that together with F-costs, obviously, because the versions of those underlying packages are different in comparison to REL-CoreOS. That is what mostly creates problems for us. Well, how often is REL-CoreOS changing within a release other than patching? You're not doing the same problems that F-costs does, just slower. Timothy can probably speak to that. Yeah, the way to fix those things is not to sign and say we stick to specific release, because that's not all F-coreOS works. So right now, OKD uses the rebuild of F-coreOS stuck to specific versions. But so essentially, you're not using something that F-coreOS tested. So the way to have this work is to put the test forward. So put the test forward in F-coreOS itself, we release ahead of time. So right now, we are trying to prepare for the F-coreOS at six rebays. So right now would be a good time for OKD to take a look at F-coreOS 36 as a base and see if things breaks in there. So that when we rebays F-coreOS 36 unstable, then OKD can consume it and not break things. So that's the way things flow. And fortunately, we cannot maintain multiple branches of F-coreOS on several versions, because we have a different way of visualizing update streams. We don't lock ourselves on versions, we lock ourselves on update testing in next streams. That's how we do the F-coreOS releases. And that makes sense. But in this case, then I'm all for going forward, but don't do it with 4.9, do it with 4.10. 4.9 is nice and stable as far as I can see. If we go ahead and add a new F-coast and that all of a sudden becomes unstable, I mean, basically it makes OKD unsupportable and unusable. So I understand that right now. That's not the whole F-coreOS works. So if OKD actually know it's content, we want to stick with the specific release for OKD. That's up to OKD. That's up to Valium, that's up to those doing the releases. It's not my decision here. But you will have to know that essentially you will be using something that has not been tested for us by F-coreOS. That's not been tested by us. Essentially, the gap here is that we have a separate build system for the OKD machine OS content currently. And we rebuild that, which is essentially a coreOS assembler compose thing that makes Fedora coreOS itself. We do that again. We run that again for the OKD content. In the future, it's called the coreOS layering effort, I think. And that is going to enable us to actually reuse what the Fedora coreOS folks have built and just layer our stuff on top of that without rebuilding the whole thing. And with that, we can then close this gap in testing where we kind of run OKD end-to-end tests already when Fedora coreOS is built. And maybe even block Fedora coreOS releases on that. And that is currently the gap. We aren't currently doing continuous testing. We are doing discrete testing when upgrading. And that is the gap. And there is always going to be that's not great, obviously. And however, we do, obviously, test upgrades. We don't release without upgrades succeeding. And that is currently our signal, whether we can release or not. And that doesn't suffice in some cases, unfortunately. But, yeah, in the future with coreOS layering, this is going to be much better. And we can say we now hold back the move to Fedora 35 and do that with 4.10. But then we'll have to, within 4.10, move to Fedora 36. We don't have to align strictly with the Fedora coreOS release. As you can see, we're currently still on Fedora 34, I think, while Fedora 35 is already nearing its end of life. So that is fine. But at some point, we kind of have to move a Fedora major version within an OKD miner. We can do that with this release, 4.9. Or we can do it with the next. I don't really care. Like, we do have the upgrade testing. Maybe we'll wait and try to get in coreOS layering before. But that is probably not going to happen before. Certainly not going to happen before 4.11. We'll probably have to do it before. It's ideal. You also need a Fedora coreOS version with coreOS layering support before you do the 4.11 upgrade. Because otherwise, you can't stage it for the reprovisioning. So we would have to move to 36 before we actually start using the capability. Because otherwise, MCD can't use it. Because you create a weird bootstrap problem where you don't have the capability to actually use it. About that. Yeah. So I'm from the MCO team. We are highly cognizant of that. We are working on that at the moment. So just so everyone knows, because I'm not sure everyone actually knows this, coreOS layering is coming in Fedora coreOS 36. Because that was already an approved feature. And it is intended to land as part of F-COS 36. So that is coming. So what we're talking about here is essentially the situation right now is OK OpenShift rebuilds F-COS for OKD. And that means that the content set is not the same as what we released in Fedora coreOS. The configuration is not the same. The packages are not the same. The testing that Fedora coreOS has done is invalidated. And that means that either there are two real solutions to this. Either one, the F-COS plus OKD tests are run on the build of that F-COS, which is not happening today. Or two, we essentially stop doing upgrades until we are ready for that transition to coreOS layering. So unfortunately, as it is right now, running OKD end-to-end tests on the Fedora coreOS config repository isn't really possible because we've set up, or Vadim has set up the OKD OS builds on an external CI system, external build system. So we can't just trigger that very easily. However, when we move to coreOS layering, we will be able to actually build that OKD artifact, the OKD machine OS content container, as part of the Fedora coreOS builds as a layered product, as a layered artifact. And with that, once that is happening, we'll then have continuous testing for OKD as well. And yeah, obviously, since we do have a signal, it's not the best signal currently. We do test upgrades. So we make a new compose. And then, obviously, test is that old compose, the old version that is already released, upgradeable to that new one. Those end-to-end tests don't run on all the platforms currently. They just run on a few of them. And that might be the biggest gap we have. But I don't think we can wait and not push out a new release for a long time. You can decide not to upgrade, obviously. Right. Yeah, if I could run. Do you have the Fcause 36 stream available yet? No, not yet. We're trying to, the rebase is happening. I'm not talking about the rebase. I'm not talking about the rebase. I don't care about the rebase. Is there a new stream available now? The rebase to Fedora 36 for the next stream is happening alongside the beta release. So it will happen next week. We need much, much to, well, two weeks. So I want to just pause us for a minute. Bruce has had his hand up for a while. So Bruce, did you want to add into this? Well, no, I just figured I'd throw in my two cents, although many other people made my points, which is that in reality, we don't use Fcause releases, right? We build our own, which may be equivalent or not. And if you read the Fcause documentation, and then you read our documentation that says we use Fcause, you're totally misled, because that's not how it works at all. And maybe that causes issues, maybe not. And it is sort of like I went through all the stuff that John went through, maybe not as bad, but I can understand where he's coming from pretty totally. But in the last year or so, I have had lots of upgrade issues. None of them have been obviously Fcause related, unlike the year before that. So, and it could be that some of our problem is that we aren't following the Fcause release and upgrade cycle, OK? We have our own, and so we don't get whatever good testing the Fcause people are doing. But I mean, there does seem to be a plan involving the layering. I've been following that all the way along, too. And so the question is, how do we get from here to there? And what I thought might be useful is that we take a lot of this and open a discussion item where people put forward the problems that they've had and the issues that they had, and then that can all get collected in one place. And we can see which ones would be fixed by which methods and how we could actually go forward. So if I, if I, I'll just interrupt from it. So on maybe on John's behalf, because I know John's on a cell phone someplace, so I don't know if he's probably in a car. So the resolution to this sounds a bit to me, and I could be the naive one here, because I don't have it in production, that maybe we hold off on going to 35 until the 410 release. So the first 410 release goes out with 35. And then somewhere in the 410 release, we do a step up to 36. Is Christian, is that what you're suggesting might be the process? Maybe that makes sense. So we can at least be safe that 4.9 remains stable, and then within 4.10 we'll have to make that leap and kind of get along. But by then we will hopefully be able to migrate to CoroS layering. Maybe we can even back part that to Fedora 35, because essentially if that lands in the Fedora 35 packages, we will still be able to do our own compose of kind of OKDOS 35, including new changes that have then landed in RPMOS tree, the base. So John, the reason why I'm trying to articulate this is so that we don't go off and have that even create a 4.9 with 35 and we hold off. But me, if I'm hearing everybody right and I often don't, that still means you're gonna have the same issues someplace in the middle of 4.10. Is that correct? My feeling is that once we release a stable release, 4.9 was released with 34, it should stay with 34. And 4.10 may release with 35 or 36, I don't know. I think once we do a stable release of 4.10, I think we should stay with it. And then, because we are focused on OKD, that is what we're supposed to be building everything on. We're not a testing ground for FCOs. So once we have a stable release, I think we should be able to keep that. And then when we do our next release or next stable or next testing cycle, then we go to the next one because I tell you, if I do an upgrade and it breaks my production environment, and yes, I realize this is open source software is self-supported and everything else, but I'm also a member of this community, and we're trying to push this forward as a way of whatever you wanna call it, doing it for free or saving money or whatever. But when we make decisions that have the potential to make the system unstable going forward, I have an issue with that. That's what I'm hearing it loud and clear. Yeah, it's also worse on vSphere, I think, because vSphere isn't specifically tested by anybody or anything. So it's definitely the production people that are using vSphere. Well, like so, I can tell you straight up, based on my own experiences here, that VMware is explicitly not tested and supported in Fedora. Yeah. So that is a huge problem. Yeah. Timothy has his hand up. Yeah, clearly, OQD is not a testing ground for Fedora CoreS, okay? We are really sure that Fedora CoreS releases are stable and tested. So once those kind of changes go out, it's down to us knowing about them, because when we know about such breaking change, we make really damn sure that they don't happen or that folks get warned about. So that's definitely not how we work. So the thing is, we don't get to fix issues we don't know about. So we cannot fix stuff in Fedora CoreS if it's not reported to us. That's why we make damn sure that we have well-advanced, well-in-advanced of the releases that we have streams with new releases that folks can test on it. So if OQD wanna step up the testing, wanna step up the stability of the project, then you have to step in and test Fedora CoreS release beforehand, like up front at the very beginning where we do them, where we try, like right now we are looking at Fedora 36. So right now is the right time try OQD with Fedora 36 intent and try to make that happen. And so yeah, there's, if sticking to a specific version of Fedora is what OQD wants to do, then it's probably not going to be using strictly Fedora CoreS anymore because that's not all Fedora CoreS will be moving forward. So typically I understand exactly what you're saying, but let me give you an example of what happened. So there was a system de-change that was done by a person that basically broke resolve.com by replacing if you were booting from a kernel boot and you didn't have a domain name, they decided that all of a sudden, well, we're just gonna put a dot in there instead of using the default domain. Well, for people who are using OQD, that basically broke their systems and we had to come up with a fix in OQD to work around that. We opened up a ticket in Bugzilla and basically the folks who own that basically said, well, Tuft, we're not gonna fix it. So you're just gonna have to deal with it. And I have that supporting documentation if you wanna see it. So I understand what you're saying, but just because you say that, if we raise an issue, it's gonna be fixed, that's not necessarily a true statement. And it has made OQD unstable in various, and that's just one example. So that's why it just, I think the problem here is- Whenever I look at it, whenever I look like I have to do an upgrade, first thing I look at is whether, FGOS is changing. If FGOS is changing, I'm like, okay, I have to go through a whole different set of testing before I can upgrade. That's all I'm gonna say. I do agree that this is far from ideal, but I think in the future, when we have that OQD end-to-end testing as part of the Fedora-CoroS test matrix, then the Fedora-CoroS folks will get those signals from OQD beforehand. I think the problem here was that by the time that issue was reported, Fedora-CoroS had already been released and everything had already been in place. That makes it really hard to then kind of, after the fact, fix those things. Well, if you know before releasing both Fedora-CoroS and OQD, Fedora-CoroS would just kind of run a new OQD release as a test and end-to-end test that as part of the Fedora-CoroS test suite. They do that now with Kubernetes as well. Since recently, there have been upstream Kubernetes tests have been added and the same will be done with OQD so that there is, so the Fedora-CoroS folks do get that signal that they currently don't get. Because currently we don't have continuous testing, we only have the discrete testing every once in a while when there's a new release being cut. It's far from ideal, I do agree, but we will have to make that one jump to get to the Fedora-CoroS layering part and after that, hopefully this won't be an issue anymore but there is going to be that one time we'll have to do it now, unfortunately. So I'm going to keep coming back to this just because I don't want to eat. So I'm still going to put on the table. So for 4.9, just to give John Fortin maybe a break in his production that we hold off and keep 4.9 and 34 and then give with a lot of forewarning and maybe some testing in the 4.10 release when layering becomes available, we move from 35 to 36 but with a lot of explicit testing on the part of this group. And I think that might be, is there any issue with that approach? So there's a couple of problems. One 34 is about to EOL in a few months. So that's like, even if, so let's say we do this, right? Let's say we do this. That means that we will need to recompose a stable release ourselves on a semi-regular basis of Fcause. So we need to recompose updates and push those out. We are not doing that right now and we would have to start doing that. Now, if somebody is okay with doing that, then sure, that's fine. But regardless of whether that is or is not the thing we're going to do, we can't stay on 34 because 34 is going to EOL in three months. So we have to do an upgrade once at least. Now, the second part of this is, there is no reasonable conception I can think of that makes any, that I can think of that makes sense for John Fortin to get a response from someone in Fedor, especially one of the core maintainer folks that are working on core system stack stuff, to say, screw you, essentially. Like he said it much more nicely, but he's not going to do that. Essentially, like he said it much more nicely, but it's effectively screw you. The only reason I can think that is that Fcause people aren't actively communicating with these core stack parts to make sure that these things are influencing those changes. And I understand from what Timothy has said over the past few months, that this is something that they've been actively working on. And I know Dusty has kept me abreast of their efforts to improve communication here. I expect that that will improve. However, it is absolute lunacy from speaking as a Fedora member is absolute lunacy that we would have a situation where someone says, this is flat out broken and they tell you, you're just going to have to stop to figure it out yourself. That is absolutely unacceptable. I don't care. They didn't say that it was broken. They said that it's working as expected now and we're not going to fix it. I think that was the system defaults though, right? That wasn't Fedora Coroizer. Yeah, but that was coming through. I mean, whoever did it on Bugzilla basically said, we're not going to fix it, even though it broke specifically what we had identified. He said, this is a change, we're not going to fix it and you're going to have to come with a workaround which is why we had to come up with that weird script to fix Resolve, whatever file it was, Etsy Resolve D, if there was a dot. Okay, can I just come in and make a suggestion here? So to me, this is not a problem with what version of F-Coffee have or what we don't have. It's the fact that we're not finding the bug and we've talked a number of times, I've been in the community coming up to a year now and we've talked several times about trying to engage community to do better testing. It sounds like if we have a more robust testing on nightlies and especially as we take say a new update of Fedora, if we had testing on cross-platform for those nightlies, we would find issues like this are much better long before they actually hit the upgrade stream. So I think part of what we need to do is create a plan of how we can leverage the community to maximize the testing we do, both while we're in this stage and we're building our own F-Coffee, but then also as you go into the sort of the streams approach, we've had this a number of times and that's the way that we actually, I mean, we could screw up within the build. I mean, we're all human. It's not just an F-cost problem. We could screw up with an OCD. The more testing we can get done on the more platforms, the less likely it is that users, whether they're just hobbyists or production, it issues. So to me, this is where we should be focusing the effort and what can we do as a community to help Christian, to help Vadim and create a more robust test regime around what we're building. I think you put that very well, Brian, and I was just gonna say, and that's kind of why the, I keep coming back to this, to keep 4.9 with 34, we're going to 4.10 relatively soon and build in, start working on getting some more of that cross-platform testing by the community earlier so that when it comes to the next upgrade mid-OKD release, we have better testing and better communication with all of the upstream communities as well. We've given them feedback earlier on. But I think that might be at this juncture, the best relief we can give to those of you who have OKD 4.9 in production is acknowledges that we haven't got the community test plans in place or cross-platform testing and work on that in the next few months and work on our communication with F-cause, System D and everybody else. And that would be in raw hide or everything else, but acknowledge that we have to do that quickly within the 4.10 release. So I think I'm taking copious notes in the HackMD, but please add in HackMD, the anything else that might, and so Christian, I'm expecting that you're probably gonna be the one that goes back to Vadim and replays this a little bit. That I just won't, I don't want Vadim to think of doing a release with 4.9, 3.5 and do that extra work. And I think the best we can do right now is that and work on our communication plans. So that said, I'd like to circle back and bring a little closure and move to the next topic. Thank you, John, for bringing it up and saying it loud again and always being so diplomatic about it. I thought that was diplomatic or not, but. It was pretty diplomatic. It could have been a lot worse. I would have used stronger words, John. So by that definition, you're diplomatic. Yeah, there you go. So thank you, Brian, also for being eloquent. That brings us to the docs update. Brian, if you wanna give us a quick update on docs, I know I have another call with you right after this meeting. Yep, yeah. So I'm gonna keep this quick. We had our docs meeting last week. We are, we have got the code of conduct. It's down to me to actually get it live. I'm having some work done on my house. So chaos runs here. So I will get it done as soon as I can promise. So that's gonna go into the okd.io site. We've also agreed to shut down the community repo. That's got five documents in. I'm gonna move that content into the okd.io site. We're making plans to create a new git org and move the content there. And one of the things we're gonna do is start to reorg the okd main repo. That's where all our discussion and our issues are. The idea is that we're gonna pull documentation out of that repo. We're gonna do it in a series of pull requests and anything that's document worthy and all documentation, community documentation that we write will live in okd.io. Obviously the product document that's in line with the OCP documentation lives on docs.okd.io. So the idea is we wanna clean up the okd repo before we actually generate the new org. So what we put in the new org is nice and neat and clean. So that'll basically be the okd repo for issues and discussions. With a front read me, which is just a pointer to where other stuff lives. And then the okd.io will be the source of all our community documentation. What else did we discuss? We now have somebody who volunteered to be our security liaison and Jamie is working with them to actually get some introduction right up so we can actually get that published. And I think that was about all we discussed. We did talk about the new operator catalog and anybody that wants to contribute that discussion use Christian's issue that's in the okd.io, the okd repo, sorry. So Christian created a repo of what operators we want to see. So that's where we should go for the discussion on there. And I believe soon there will be some discussions that'll be reported back on that issue. And that's all I have from the docs meeting. And that person who is the security person is Mohamed who is on the call right now. And Mohamed, did you want to make a comment? Yep, you might be on mute Mohamed. Oh, he didn't get a contact from Jamie. Because Jamie is on vacation in Florida this week and good on him. So Mohamed expect something next week. And if you don't hear, come to the docs meeting and we will hijack that and make sure you get what you need. Okay, thanks. All right, let's see. I'm going to try and share my screen again here and walk through any issues that we should do quickly. Because what do we got for time left? We got 10 minutes, so good on us. The issues, open issues. I know Charo, update OKDBase CRD build. Charo had brought up the issue in Slack somewhere over the past week about not continuing doing CRC builds. I'm not sure if that's exactly what this issue is about. But I wanted to mention that on this one to see he's right now, the same person he's right now, the sole volunteer on the CRC build process. We were supposed to get a talk on micro shift and Charo's not here. So I'm going to let you guys continue this conversation. It's not that it's dead. It's that the community has an investment as he puts here. So just so that's on people's radar, if you have, I have bandwidth and want CRC builds, we need volunteers. And so reach out to Charo so that he's not the only one doing that, because I think from the conversation, he's doing them, but he's not using it. So he's gone to snow. I think the conversation was that CRC is too big to be useful by a lot of people. And we were hoping the micro shift an OCD version of micro shift might be a better option. So I think that's where we were hoping to have that conversation. So we can make the decision to keep supporting CRC or actually formally place it with a micro shift. At the top of this deck, the OCD is the slides for micro shift. I thought someone was going to give a presentation on that today, but the slides are there. My understanding is that micro shift is built with OCD in mind. So I think we already have that. It's just whether people are using it. I think a few people have said that they've played with it a bit. I like it conceptually, I have not played with it. So I'm going to try and get Sally or one of the two folks who are in Spain who are leads on that to show up to the next meeting to give a presentation on that. But that was that issue. I think that was the major new thing on the issues list. And other than the operator wish list, which is the perpetual thing on the wish list issue. And if there's any updates to that, Christian, I don't think there's an update. And then there's the discussions and idea stuff. And the online assisted installer service for OCD was the latest one that I saw. And that's two weeks old too. And I think that's something that Vadim has been working on if I'm correct, Christian. And he's not here right now, but if you want to pile on to that. Sorry, could you repeat that? I was- You were distracted, never get distracted in no KD meeting. I'm just going to share my screen again. I do have an update to share though. So I've just chatted with Vadim and he will keep OCD 4.9 at Fedora 34. And 4.10 will be moving to 35. And then we'll have to figure out whether we will either do the move to 36 within 4.10 or we skip a Fedora major sometime later on. All right, cool. Those are kind of the two options. Maybe it might make sense to just skip a Fedora release. And well, we'll see about that. But I'll definitely, for now, 4.9 will stay on 34, 4.10 will move to 35. Okay, postpone the pain. Yeah, thanks for doing that. That's a worthy distraction. And I'm just looking at the discussion list. This is the most recent one, the online assisted installer service for OCD. I think that was something that Vadim was also... Right, so it is possible to kind of deploy that. And we have a kind of proof of concept for running OCD or installing OCD with the assisted installer. And there is also the assisted installer is kind of two pieces. It's an agent that runs and it's also a web service that kind of generates an image for you and the images that are then installed call back to that web service and you'll kind of get a status update and as a visual on that website. And that is kind of in the works, but we're not sure and that is, it's even an open shift, it's just a tech preview and we're not sure whether that is going to be productized at all. So we might not go through that, but we're definitely focusing on this agent driven install, which will make it much easier to install open shift in general and OCD obviously as well, on any platform without having to do too much integration work that way. So that is a focus of ours having this agent driven install whether we need the web service part of it or just make it kind of a CLI thing. We don't really know yet. So I would just say stay tuned and watch that issue if you're interested in that. Is there anything else in the discussion or the issues list that people wanna raise and discuss today? We get about four minutes left in our hour. Sure, yeah, I wanted to point out that we have the discussion about what the purpose of our install guides are and how kind of what format they should be in. I've raised it. I haven't gotten any feedback from anybody yet who's previously participated in the guide. Brian and Jamie, I think are here. Jamie is on vacation. Brian is here. Can you throw the discussion URL into the chat just for me? Yeah, it's in tasks right now. I'll ask to get in the chat. So I don't know if this is a case of everybody likes my proposal so nobody felt the need to say anything or people haven't had a chance to read it yet. I'm not sure how long to go without just saying, okay, nobody's objected. So the thing I wrote up is the thing we're doing. Right. I totally missed that. I was out with COVID during that time. So I do think, I think I like that. And most importantly, I'd like all the guides to be to share one common format. So it's kind of clear how to add another one and what the actual purpose of each guide is which should be the same purpose platform, I'd say. Yeah, I do, I've just had a very quick look, but I'll review it a bit more thoroughly in a bit. Thank you, and I hope you're feeling better. Yeah, I had three shots and it still hit me pretty hard, but yeah, definitely better now, thank you. All right. All right, anything else anyone wants to bring up in these final few minutes? Otherwise, thank you all for participating today. I really appreciate the feedback and thanks for stepping up there, Christian, and doing the back door discussion with Vadim. So we got that update in this week. There is a docs meeting next week on Tuesday. And the other thing, well, under the docs kind of thing is we're looking at staying with MakeDocs, staying with okd.io, but using a bit of outside contractor help to do a refresh of the okd.io site. So it's a little more, you know, perky or websitey or whatever it is. We have some dollars in an account somewhere that I need to spend. So Brian and I are gonna meet with Brandon right after this call and discuss that. So we may be bringing back a few wireframes and suggestions for things to make it look a little bit better too. So that might help in the usability of the site. So with that, I'm gonna let you all go back to your day jobs and thank you all again for all of your support and talk to you all soon. Thanks everybody. Take care. Thanks Timothy.