 Hello, YouTube. I am so sorry for that delay. The YouTube backend did not want me to stream, apparently. It was being very grumpy, but now we're live. Hooray! Uh, Neil, are you with me? Yes, I'm here. Although, uh, that lag is interesting. Are you watching the stream and the call? Yes, I am. I'm in both places. Just like, you know, everyone else seems to think I am. Yeah, okay, so now we can actually do what we came here to do. So, today is Wednesday, April the 7th, which means that it is Fedora 34 Beta Test Day for upgrades. So, what that means is this is the day where we're supposed to test going from Fedora 32 or Fedora 33 to Fedora 34. And that's a lot of 30s. So, there we go. Um, YouTube chat. Tell me if that, oh, okay. Tell me if that screen is big enough that you can read what it says. So, this is, um, the website where we have all the different tests that need to be done for upgrading. Um, so there's the basic updating with DNF and then doing that with encrypted, um, unencrypted file system. And then there's doing it with GNOME software and doing it with older versions or doing it with Fedora 32. Um, what I'm going to start with is just doing the basic updating from Fedora 33 to Fedora 34 using DNF. So, I've got a virtual machine where I should be able to run it if I can ever figure out what I'm doing. See, this is my first time live streaming if you couldn't tell. You're doing a fantastic job. Like, I'm impressed with the fancy that you've put out. I'm not. But at least one of us is. Okay, get rid of that virtual machine manager. Okay, so right there we've got, we've got a normal freshly installed VM with Fedora 33 workstation. So my password is the most secure thing I could think up, which is password. The best part is that this machine is not open to the internet. So only I can access it. So let's get out terminal do a neo fetch just to show off our Linux enus. Oh, I don't have that installed yet. That's a problem pseudo, you know, before we let's see test case upgrade DNF current workstation DNF current. So the prerequisites perform an installation of Fedora 33 workstation with default partitioning. Done that. Do a full system update and reboot. So let's do that pseudo DNF update. I have received the usual lecture from myself. Are you sure you properly lectured yourself? Okay, we can double check now. I will respect the privacy of others think before I type. Well, I don't know about that. And with great power comes great responsibility there done. I don't know what that I feel satisfied by this, but okay, let's go on. Well, it's done. The networking on this should be fast enough to veto Fedora 34 is available now, but it's just a beta. So you shouldn't run it in production. You should only run it on systems where you're comfortable with. I'm not going to say like it's going to explode, but there's no guarantee that it's ready. It's called a beta because we're still testing it out, but you can get it now. Oh, thanks, a doctor. I spent about the whole day on it. Yes, neo fetch needs to be in the base Fedora packages. It is needed on every Linux system to show off. No. Yes. No. Yes. What's your problem with neo fetch? I have no problems with neo fetch. Other than it drags in like a dozen extra pearl things and I don't want pearl things on in the base installed. I don't have to. Sounds like a you problem. That satisfied me. There's always so many updates on a new system. It's right, Neil. Pearl is your friend. Listen to the chat room. I don't think people have realized how much I've suffered. So once this is done, we're going to need to figure out how to on full screen. There we go. That's where it is. After this, do a reboot. Install the latest stable version of DNF plugin system upgrade package. Download the latest updates for 34 to aid debugging run this command to enable a root shell on VT nine. What is VT nine? VT nine is virtual terminal nine. Like if you control alt F nine, that takes you to VT nine. I've never heard of it. Hold that. I was expecting it to be. Oh boy. Oh man. Yeah. So yeah. So TTY's teletype terminals are actually virtual teletype terminals or VTT wise, which we just call them as virtual terminals VTs. Um, actually in, in most, uh, the actual subsystem, I think they're actually known as PT wise, which is pseudo terminal, uh, pseudo teletype terminals, which whatever. They're fake. They're not really teletype, but, but that's where everything is built on. So that's what it's called. Hey, Robin, I haven't seen you in a while. Have you done the beta test yet? So right now we're downloading the packages or what? All 831 of them. Are you sure you installed from the live CD? Yes. Well, I didn't install like the library spins that are constantly made. I installed the last. Okay. You started from the GA version. Yeah. I'm doing the whole follow the actual instructions thing, which recommended I do that. Oh, wow. You're following. Oh, okay. I know. I don't normally do that, but I decided if I'm going to like do something relatively important, maybe, maybe I should follow the instructions. You don't follow the instructions for your own home server. I don't want to talk about that server. Actually, the VMs I'm running are on that server right now. So maybe a bad time to bring up how janky it is. Hopefully it won't crash on us. Oh, it'll be fine for probably the length of this and immediately afterward it'll go and be like, I don't like you anymore and then die. It hasn't crashed in a long, long time. Both of our social should be in the description. I don't have a social. And Neil didn't ask so. Wait, what? Hold up one second. No, I won't RTFM. Don't worry. Yeah. Hammer knife. If you are running on a well, we if you're running on a fedora 32 or before install, your partitions are probably ex t4 and just upgrading isn't going to repartition. So you can fresh install it or you theoretically can change your partitions over. I don't know much about that process. Um, so if you want butterfs, I personally would say just reinstall. Um, if as if you're still running an ex t4 partition. What happened that, you know, door to door salesman life. Um, so what it's a quarter of way through the upgrade right now. It looks like we're on about three and a or three hundred and a half packages of 1600. How come there's more to upgrade than to hold down? Ah, so this is interesting. There is actually two phases to this. So there's the actual upgrade and then there's the verification steps. So the actual upgrade step is actually applying the packages and installing them and running the scripts and stuff. But then after that's done, it runs through a verification step. It's effectively, if you're running it on your system, RPM dash. RPM dash, I think V on RPM dash capital V lowercase a, which does a verify of the entire system. And if it doesn't see anything too outrageous or out of the norm, then it passes and then it marks the transaction has succeeded. So that's why the numbers exactly double the number of things. Package management. Very simple. Okay, Neil's a perfect person for this topic. What are the benefits of butterfuss? This is an unbounded answer. Well, specifically for most people, the main advantages of butterfuss are going to be under the hood. You're going to have a file system that has better optimizations for modern storage, including handling flash storage more effectively and dealing with high density classical storage like hard drives better. You know, you'll have the ability with the copy on right stuff. There's a some level of integrity protection as you're doing data rights because it does check something of the data as opera right operations and read operations occur. And of course, because of all these features that are built into butterfuss, you have things like sub volumes and snapshots and things like that. And that allows you to do things like you can do a thin partitioning of, you know, partitioning of resource allocations in the file system without making them a hard like real partitions where you are dividing up space. So you have the root operating system sub volume and then you have the user data sub volume for home, and they share space, unless you make them not share space. And that is your choice and for most people that means that they have full access to their storage and it's much more efficient that way. And so it's a foundation for a whole bunch of other potential features like with sub volumes and things like that you can also do snapshots so you could preserve the state of the file system at any given time. And with features like butterfuss send and receive you could actually transfer those snapshots off site, which can be useful for replicating systems doing disaster recovery scenarios all those kinds of fun things. And you can do them efficiently because it's at the block level rather than the file level. Some of the other bits around copy and write and stuff allow for a feature called ref links, which means that, you know, when you do things like copying files from into another part of the file system. It's effectively a zero copy operation where the data blocks get completely reused and then as you modify a file, it will only allocate new space for changed blocks and the rest of it will be shared and remain the same. So it's much more space efficient in that regard. So you get all kinds of interesting things. Because the ref links think of you basically in bandit duplication sort of sort of in bandit duplication and things like that. So it gets it's, as I said earlier, it's an unbounded answer. There's it really depends on what what you're looking for, but there's just generally benefits across the board, no matter what you're doing. Definitely. It's, it's a much smarter file system than something like EXD for. And long ago, people position butterfests to essentially replace the exed family of file systems. At one point, I believe it was said you could consider it exed five. And a lot of the under, you know, a lot of the evolution of the exed family of file systems. It's kind of related towards that path. And butterfests kind of take some of the things that people have been trying an ex in the exed family with x four to its logical conclusion where if you're using extents which is data type data structure used in file systems. If you use extents for basically everything, which is what butterfests does it uses v plus trees with extent trees underneath it to for everything. And that is, that's part of what's made it so difficult to design a file system this way because nobody's ever done it that way before. But after, you know, decades, it's finally, it's been in a position where it's fairly solid and and it works well in both the enterprise and the home user case. Thanks for the explanation. I'm locked. Ali, I believe that we want to do that eventually. But, you know, don't want to change too many things too fast. I actually do have that running on my system. I set up what's I set up this program called snapper, which hooks into DNF and takes a snapshot before and after each upgrade. I think that someday we want to do that in Fedora. So instead. It's also just hard to do because the way that bootloader works in Fedora is different from every other distribution. And our VM is updated. So the next step is reboot. Yay, exciting things are happening. Okay, so systems back up. Let's go ahead. Oh, wow. VM double keyboard bindings are confusing. Pseudo DNF update just to make sure there's no stragglers. There shouldn't be, but I don't know. Seen that on some systems before. Raging Goon, are you talking about for a butterfess? Because in that case, I don't think it should have any effect on code compilation. So. It depends. It could. But just about as much of an impact as any other file system would, like, if you are trying to build compile huge code bases like Chromium, you probably want to do it in its own butterfess of volume so you don't run into iNote exhaustion and other fun things that could happen when on an X4 file system because you're compiling hundreds of gigabytes of things. And it's just, it's gross. So our next step is to install the latest version of the DNF plugin system upgrade. So is that a plugin for DNF to allow you to do something with upgrades? What system upgrade? Yeah, it's a it's a plugin that implements offline updates through DNF. And so the way it works is that you pre download everything with DNF system upgrade download. And then when you do when it's validated all those it's verified everything it's imported the GPG keys, then you do DNF system upgrade reboot it takes you into a minimal environment that applies the depending transaction. And that gets done in a minimal environment so that it doesn't potentially cause random breakage in your system. This is the safest way to apply major upgrades like this. Even though it is possible to do it live, like if you want to do it while your system is running, that's essentially DNF distro sync up to the new up to the new version. It gets very dicey, especially with things like GNOME and plasma where there's a lot of interdependent components that can just fail during this. So the offline upgrade mechanism provides a much safer path to doing major upgrades like this. Safe is always good. Confirmed. So now we're moving on to pseudo DNF system upgrade download refresh release version 34 so we're gonna switch everything over to Fedora 34 push the why wait for it to pull down the repos compression is great in response to Ali. Oh neat. That's an unconventional way to use it. Yeah. So one other thing that's interesting about butterfests is that you can set different compression levels and different compression algorithms on a per file or per folder or per subvolume basis. So while the change in Fedora, in Fedora Linux 34 is that on fresh installations, you're going to have Z standard compression at level one. You could choose that if you have a particular subset of your file system that needs to be level five, level seven, level 14, level 22. Well, if you want to change it from Z standard to say LZO or some other algorithm that it's supported, then you can. And it's fairly straightforward to do that. Wow. This is a lot of packages. I mean, I guess that's to be expected but it's a lot of packages. No match for group package xorg font things. What do these mean? Do you have an idea? Since it's not throwing. I'm lagging behind you. I don't know what you're looking at. Oh yeah. So what you're asking about the no match for stuff. No match for so. Yeah, so what it says is no match for group package. That means that the package in question that's listed in the composition group that was being processed for the upgrade does not exist. The package in question does not exist in the repos. It is however not a hard failure because the group doesn't say that it's mandatory. So some background here. We have this concept of composition groups or a lot of people call them comps groups. You can see them by doing DNF group list. And you will see all the different groups. You'll see which ones you have installed, which ones are available and so on. When you do a DNF system upgrade it processes those groups and attempts to add them to the transaction to make sure that you everything gets upgraded correctly even with the groups. What's happened here is it says well these packages are being requested effectively like a recommend style, but they don't exist. But because it's only like a recommends rather than a requires it doesn't hard fail and it just says what we're going to skip them. So it probably won't cause any consequences. It probably won't cause any issues. Those packages, it's more likely that those references have been there for years and years and years and packages themselves have been gone for a while. And so you probably don't even have them on 33. So just getting cleaned up? You see it would actually hard fail if the packages were installed on 33 and it didn't have an upgrade candidate and it didn't have a way to get obsolete or if it didn't have another way to resolve the transaction. But that wasn't what happened here. This was it tried to request a new package. It doesn't exist. It's not a hard dependency. So it skips it. So out of curiosity while you were talking I did a DNF group list. We've got lots of groups or a custom operating system minimal install server edition so all the different spin type things and then different server. So back on topic. Scroll through a massive amount of packages. Get to the bottom. Oh, no, come on. Get to the bottom. Okay, so all everything looks good. We're going to upgrade at 1692 packages. Install 74 and downgrade to that started up. What are we downgrading? I don't know. I don't know if there's a way we're downgrading OS tree. I see it. It says downgrading OS tree and OS tree live so that's not a problem. Downgrading RPM OS tree. Not RPM OS tree just OS tree. What they're not the same thing. Oh boy, you get to learn a new thing today. I've learned five things from this stream alone. Yeah, so OS tree is the foundational technology that is used for content addressable storage for root file system tree. And that's what's used in flat packs. That's what's used in RPM OS tree for the operating system. And so these libraries are used to support flat packs on here because the door workstation ships with flat pack support out of the box. Damien, I hope I pronounced your name right. Upgrade update thing are kind of a legacy thing where previously you had to update which would pull down your new repo lists and give you a list of what packages need to update. And then upgrade would pull would actually pull down those packages and swap them out. Nope. No. What? Nope. Well, that would that not only does that terminology still not make sense even though that is that's how it works in the Debian world. That is not actually the problem. That is not the reason that those terms exist. The reason those terms exist and why their synonyms now is that in young DNS predecessor, they were different. In the battle days with young young update and young upgrade did slightly different things young update was essentially the equivalent of what was essentially doing an update where it wouldn't process obsolete. So package replacements would not happen. This behavior doesn't even exist in the latest version of young that is shipped in in rel seven. But this was what the original terminology was young update did just upgrades of the packages so it's essentially RPM VH. It didn't process obsolete so package replacements didn't happen young upgrade processed obsolete so package replacements would happen. The analog here is if you're familiar with the Debian world of apt get upgrade versus apt get dist upgrade. The difference between the two is that apt get dist upgrade or if you're using the newer apt command at full upgrade apt upgrade does package updates and does but ignores breaks and replaces in package metadata. If you do dist upgrade or full upgrade it applies the rules for those and does package transitions. That's the difference and in DNF early on everyone decided that it was dumb to have this decision have this split and most people don't understand what that split even means. And so we always process obsolete so package replacements always happen and update and upgrade are now synonyms. That's what it is. Glad we have it unified now so I don't have to think about this that is complicated. It was also stupid. That's why I have Neil. It was also stupid. That's why I have Neil. Okay, are we back now? Yeah, I think so I saw it blip and die and now it seems to be okay again. Yeah, I was wondering why OBS studio was waiting so long to crash but it was also stupid. RPM fusion is it so pro boss 2003 is asking is RPM fusion built enough for a trouble free upgrade. My personal laptops have RPM fusion activated on them. I was able to upgrade. Okay, your mileage may vary. It is a third party repo. The Fedora project can't guarantee anything about third party repos. That is why we don't test with third party repos. You know that the parts of the RPM fusion repository that we include by default on Fedora workstation are ready to go. And in general, the RPM fusion team does a good job these days of keeping up with Fedora raw hide the rolling development tree, as well as Brent releases and development releases such as for example right now Fedora 34, Fedora Linux 34. And so I don't think that this is actually a problem and you should be okay. We're now on to accepting GPG keys so we're loading the Fedora 34 primary GPG key. Pressing why these transaction tests always weird me out because oh, that I was about to say they always take a while and it can I'm like it doesn't show an active log of what it's doing. So I know it's doing stuff but I can't see it. But then it messed with me as I was trying to say it and completed fast. Okay, so and raw. I don't know how you want me to pronounce that but when you upgraded VLC was giving you an error. I don't know if that should be filed as a bug with VLC or against Fedora 34. If it needs to be filed against Fedora 34, I had a link somewhere. Yeah, I think that would probably need to be filed against VLC. What do you think Neil? Neil, did you leave? You might have lost Neil. Yeah, I've got the best seat in the house for Internet connection. I'm using the VMs are hosted on a server that's got Ethernet directly into the router and it's it runs the DNS server so it's got fastest access to what it needs. Okay, um, back to the guide we just did the system upgrade download refresh. I'm sorry I didn't hear the thing you were I heard you say you asked me something but I didn't hear anything else, because now you can hear why. What's that? Did we lose you again? I'm confused. Okay, so I'm going to guess that the job is calling. So we've done it was trying to try to you scared it off. So I think according to the according to DNF, the next step would be to the next step would be to do a DNF system upgrade reboot. But I think we pseudo system TCT we need to do the virtual terminal nine debug. So to aid debugging run this command to enable a root shell on virtual terminal nine during the upgrade boot. So the command is not that pseudo system CTL add dash one system dash update dot target debug dash shell service. So we're adding system debug shell. That's done. I actually don't know how. Oh, okay. So I guess I can send keyboard combinations with the withvert manager. So we're going to do now pseudo DNF system upgrade reboot. So this is where the whole offline update thing kicks in, I believe, because we've got the packages installed. And now we're going to do an upgrade reboot. You only got it in one. Cool. So to run it. That was weird. Do not turn off your computer. I'll try that would be unfortunate for the stream if I turned off my computer. That's right. Fingers crossed. I hope we won't have to file any bugs. But if we do, I get to learn how you're timing it. I know how to file bug reports. I know how to file bug reports. I've never used the bugzilla specifically. Oh, okay. Yeah, I've typically filed all mine and, you know, pager, GitHub, those type of places. This, this screen makes me very happy just as just just because it's very like modern. It's clean. It's clean. And it's, it's very clean. It's very simple. And it's very comfortable. When people see it, there's no reason to be nervous. There's no scary text going on. There's not gibberish on screen. It's just simple. This is the kind of things that makes me feel like the door is super polished. Like, you know, other competing operating systems. Like this is a kind of screen that you would, I would expect to see on, you know, a Mac or when, when doing a system upgrade on a Mac. So the fact that we have something that nice is really, really great. Robos, I'm sure I could get the nasty text, but do I want to see it? If you really want to see the nasty text, press the escape button and you will see it. If I press it again, will I get out? Yes. Okay, let's do it. Ah, it's horrifying. Wow, it is moving super fast. Let's not look at that. Robos, because it's just so shiny when there's no, when there's no scrolling text. Everybody in chat's like, no, scrolling text for life. Y'all are the wrong people that this is for. This federal workstation. Edward has something great if you don't want all the shiny polish of Ganon. I'm going to guess he's going to bring up that, that thing that's called a tiling window manager thing that I3. Hey, don't be mean to I3. I like I3. But yeah, as Josh lay points out, the console spam actually makes it take longer because it has to process. It has to print out the text, buffer it, log it and send it at the same time. And so it actually takes more work to show it on screen than it does to do this. The upgrade does go faster when you're doing it this way. Just buy a little bit, but it does. And it's happened. I3WM forever. Sway though, sway though. Yeah, come on getting on that Wayland life. Now that the, that now that the driver that shall not be named is actually going to support the proper like framework for working with the, with the graphics subsystem. Sway will work with that too. Everyone's happy. Desktop should not use more than 100 megabytes of room. I don't know. The wobbly windows. I mean, how can they call it sway if the windows don't sway. Exactly. That's sort of the point. In any case, somebody says it can't modern desktop should use no more than 100 megs of RAM. Well, I'll say to you this. Lex cute or LX QT 90 megs of RAM. Done. Got him. I'm sure you could slim Katie down to that. Yeah. If you actually, if you take a canadi out and then you and the rest of the Katie him stack, which means you're not running in my SQL server in the background that then I think plasma is like 140 megs of RAM, which is really, really small for how feature rich it is. You can turn on and off all the things in plasma. Yeah. And I think if you really worked at it, you could get plasma to be under 100 megs of RAM, but that sacrifices a lot of the quality of life. A lot of the wobbly windows. And the wobbly, wobbly windows. Actually, I'm not sure the wobbly windows would be sacrificed in that regard because the wobbly windows run through run through the depositor. And now that we're Wayland by default. Yeah. Wayland. It means that the all the comp compositing stuff is actually offloaded to the GPU. I feel like I should have like boosted your volume really high when you said Wayland. No. I already know that I'm very loud on on the other end. Are you? I can turn you farther down. Yeah, Lux cute is real nice. I use it for a lot of my virtual machines because I I prefer having a graphical environment for my VMs and the Lex cute environment is super lightweight. And it gives me just enough desktop for for me to be comfortable with things. The other one that I'll occasionally use is XFC, but XFC is just it's not that lightweight compared to Alex cute. It takes like 200 megs of RAM. And when I'm spinning up VMs that have two gigs of RAM total. You know, I really like to have as little RAM used by the desktop environment as possible. So yeah. Perkin, that is a good question. Does Lux cute support Wayland? I'm gonna Google that. Unless Neil knows. I'm not entirely sure. I think because it's just built on cute and the kf things. I mean, I guess maybe I'm not really sure. I know that if you switch from open box, which is what the Fedora Lux cute spin uses now to Quinn, which is what the Katie spin uses. The session can run as a Wayland session. So the desktop starts up that way. But I don't know if that also means that you get all the applications running as Wayland native applications as well. So I guess I don't know. So I found an open issue for it. Let's put it up on stream real quick because I can do pro streamer things. Hooray. You're so pro. So pro. So this ticket's still open. So I'm guessing that it doesn't fully work. It looks like it's been open for seven years. Oh, that's unfortunate. Yeah, it looks like it's mostly just under discussion like how they even want to do it. Well, that's too bad. Wayland is definitely the future. That's for sure. If not now. The sad thing is that I couldn't be running this stream under Wayland because of OBS. It doesn't support Wayland well enough yet. Well, you could have because of a thing that I have, which is actually you could have because the flat pack from FlatHub is OBS 27 RC one, which is which it has Wayland support. Oh, really? Yep. Last I checked it. And also I have private builds that I've made for myself of OBS Studio to kind of play with it being Wayland. So I've got private builds of OBS Studio based on the RPM Fusion spec file that have that are built off of the latest commits to try it out when it works okay. Do you use OBS much? I have streamed a grand total of two times. What do you think? You inspired me. I wasn't that good. You were good enough to inspire me. That's something. Well, you have surpassed me by a wide margin. I don't know about that. I just had a dumb idea one day and managed to talk to the right people to make it happen. But yeah, of the major desktop environments with Wayland support, the one that's probably in the most hot water is Cinnamon. Cinnamon is probably the one in the, well, there's also the dead one, LXDE, but Cinnamon is of the ones that are not dead. Cinnamon is probably the one in the most trouble because Cinnamon upstream has done absolutely nothing to move towards Wayland in seven years. As far as I'm aware, they don't even have an issue open on their GitHub for it. How's KDE's progress on Wayland going? The only, the kind of, there are no deal breaker issues as of right now, at least based on conversing with the KDE developers and working with what we've got. And some of the remaining issues that we have currently are intent are will be fixed in upcoming bug fix releases or the next feature release coming out in June, class 1522. But the remaining issues are really GTK integration issues where GTK is not using the correct Wayland protocols or in the case, in some cases, the Wayland protocol hasn't been proposed and standardized for both of them to use it. So once those are in place, then the remaining issues are basically gone and we're at parity with what we had before. And I foresee that happening within a couple of months. The fact that Fedora will be shipping Plasma with Wayland by default has been a great driver for getting the KDE community to get things over to the finish line for Plasma Wayland. And I've been a part of that. I've done a fair bit of debugging, done some cleanup work and some fixes here and there and file bug reports and things like that. And the KDE community has been fantastic on this and very responsive and very helpful and very friendly. So I've been really grateful for all the work they've done and how much help they put in to helping make this successful for us. Awesome. I just switched back into the speedy text and then back out. And I don't know if I broke something, but now the text at the top is gone and we've got a new logo. That's interesting. You didn't break it. This is what happens when you do live updates. And that's why we're live streaming is you can see a break in real time. Fortunately, it's not damaging. So what's actually happened is that the branding has changed. And the way that Plymouth actually works is it constantly refreshes from disk the screen as it updates the frame. So like every time it moves the progress bar, it's actually refreshing the whole screen. So that's why you get the new logo now because the new logos what's on disk. Yay, new logo. Though that is certainly a very fun and interesting experience. I went into the scrolling text and it was just like moving at a million miles per hour. So this is Fedora 33 workstation, right? So what other ones do we have planned to do an upgrade on in this session? I don't know. I have no clue how late I'm going to stay on here. I have both a 33 and 32 cloud image that are waiting for me to do whatever I want with them. So we could play with that if we have time. We could have also been doing it while this was going. Oh, we could. That's a good point. Hey, we've still got time. We still can because it's going to take a while. Okay, let's take this. Let's make it, let's make it tiny. You're so pro. Oh, thanks. Oh my gosh. There we go. Okay, so, no, go back somewhere. Here we go. We've got, there we go. Nearly. No, go, you go down side, downside. Scroll. Ha. Okay. Maybe that's working. Tiling the KDE Plasma way. Almost as good as the i3 way. Oh, got him. Ouch, that was very loud. Sorry, couldn't help myself. So now we have this. I don't know how visible this is. You're right. Ali YouTube channel should. So this is a droplet I've got running. It's running for our 33 cloud edition. And then over here we've got identical specs running 32. So let's go ahead and run through the gamut with them. Test case upgrade DNF current workstation. So since it is not workstation, I don't, I assume that everything should be the same, right? Yeah, so the DNF system upgrade method works for all variants of Fedora. Fedora workstation has an additional upgrade method through GNOME software. But if you're looking for the universal way, just do it with DNF system upgrade. Now, you will have a fun little consequence here. Since you are doing this with Fedora cloud, we'll probably want to have the console open from the, from your VPS provider. Because you won't see anything while this is happening otherwise. You'll just kind of have to sit there and wait. Really? When the actual reboot, because, because when you do this, the reboot, right? When you do the reboot, it's going to go into that minimal environment where you have no shell and no SSH. It's going to do the upgrade. It'll probably work, but you won't be able to see anything. But is there something to be said for scaring the crap out of us? It makes for great content, I suppose. Hmm. We'll do the air. Let's see. I'll get the dashboard thrown up on my other screen, just in case. Or when it reboots. But let's go through the thing. Pseudo. DNF. Update. Go. Let's do it on both of them. Pseudo. DNF. Update. If you want it to just do it without prompting, you could add dash, dash, assume yes to it. I could, but that would require me to think of it. Also, why is your text green? It makes it look awful. Because it looks cool. No, it doesn't. Green is terrible. You're mean. Can change the theme, though. You can leave it. It's just like, I'm a sad because like, because it's green, you can't even see all the fancy colors I added to DNF. Or this version of DNF might actually be sold. It predates my change to add colors to the output. Yeah. I don't know. It will be, but when you do, once the update is done, you will have colors. There we go. Does this make you happy? Well, I can't tell because right now it's, ah, there we go. Yes, I'm happy now. No. Ha. Stop it. There we go better. Wait, Neil, you're Irish? Wait, what? Someone said Neil's Irish is bleeding. Pretty sure you're not Irish. I do? Ah, I don't. What? Hey, we could all... I'm not an Irishman. Huh? I'm not an Irishman. You're not an Irishman? I've never been to the land of here. We could always get cool retro term if we needed to. Oh, no. Come on. No? Really? You sure? No. Ah. No. It's slow and cool looking. Workstation is rebooting. Come on, password manager. No. I feel like I'm required to type my password three times wrong before it will actually let me in. Don't password manage your SSH keys. You should know those. Effort? Lissai. Lissai. Okay, so now we are back into the machines. But for now, let's check out this. What is happening? Oh, no, it's just booting. We're good. Okay, so let's get logged in. Work. There we go. And now. Welcome to Fedorith Linux 34. Hey, look at this. Look at this. Welcome to Nome 40. If you want to learn your way, check out the tour. I would. Essential new features. Get an overview. Ooh, look, it's round. That's new. I'm sad though. We don't have the animated video thingy that shows up when you start the tour. Like the one in Fedor Linux 33 just made me like tickle with joy because it was just, it showed the Fedor logo and it did this little super fat, the speed animation to fill out the infinity. And like, I just, I was just so happy about that. But now we don't have that. We just have these plain picture thingies. My touchpad gestures don't work over whatever I'm using. Well, yeah, you can't do that unless you pull the touchpad device into the VM itself, which you probably don't want to do because that means you can't use it on the host. Well, I do. I mean, I'm using an external mouse right now. Ha, ha. Look at this. Oh man, there's no fancy animations. Look at that. Okay. Welcome to the joys of not having a hardware accelerated graphics and virtual machines. I just had to, I just had to make some folders. Look at this. I can shuffle stuff around. And I'm 40. Everything's working good. Nice. Everything is looking nice and polished. Oh, look at that expose thingy. It's floating outside the workspace. I'm 40 is thinking outside the box. Get it? Oh, Grayson. No. Hey, I had an opportunity. Had to take it. Okay, so let's do this. Close that out. System reboots. Should reboot, new release, grab menu. Test basic applications. A terminal file browser and other depending on the system by Frobos. Thanks for coming. I hope we can do these streams more often. They're actually really fun. I'm glad that I didn't like completely destroy something. Okay, let's try out a terminal because there's something you have to do to desk and I'm 40 and that is neo fetch. Oh, look at that. It's it asked me for the package because it wasn't installed. Nice. I can't remember what that is called. That's just so satisfying. Because it's it's just very polished. It's called an integrated experience. That what it's called. Oh, look at that. Fedora 34 workstation edition pre-release. Ouch, pre-release. Okay, then. So is there anything else we need to do? This is look, I think it worked. I'm not seeing any errors. Why would it fail? I don't know. It's a beta. But you're assuming that it would fail? Not assume that it would fail. I just accepted that there was a possibility of complete and utter failure. It's a beta. That's part of what beta is. But it worked. So that's all. Anyone have any ideas of things that I should try to see if it will see if I can break it? Play videos in it. The YouTubes. Oh, the YouTubes in a VM over the network. Seems like a terrible idea. Let's do it. No. Well, make sure you play something from the Fedora YouTube channel. Hey, let's watch the live stream on the live stream. I like it. Let's do this. This is going to be terrible. This is not going to go well, but let's go find out anyway. YouTube.com slash Fedora project. YouTube.com slash Fedora project. Fedora project. It's actually going pretty fast. Hey, look, that's my face. It freaked me out at first. I was like, why is my face on YouTube? Because you didn't set a thingy. Oh, look at that. Look at that. Oh, this is so good. Hey, it's like fairly high FPS and everything. This is great. Now, what about dem audios? Dem audios. I don't think I can stream audio with the way that this is set up being a VM over the network and all that. Especially since that would totally mess it up since it would be hearing my audio into. That's confusing. Yeah, sure. But like, how can we tell that it's actually playing any audio? Like, can you see the waveforms as they're coming or something? We could. Is there like some sort of volume up that'll let me see the levels? Sound system volume. Maybe the sound thingy in the settings. Oh, it worked. I heard the stream somehow. Okay, there we go. So that worked. I don't know if you guys will be able to hear that down on the stream, but I just heard the stream through the VM. Success. Hey, look, responsive UIs, they work. What do you mean? Like the snappy? Well, when you tiled it, the UI for this GNOME settings app looked changed. You're right. It did. Oh, fancy. Polish and fancy. Nice, much nice. Come on, do it again. It's not good. There we go. Cool. Okay, let's stop this because it's becoming YouTube inception. Let's go check on those cloud boxes again. Ah, where'd it go? Oh, that was interesting. It actually, when you were talking, the line in actually was going into the VM. What do you mean? When you were talking, it was showing that the microphone audio, the line in, was showing audio samples coming in. Like it was, there was a level showing off that there was audio coming in while you were talking. Oh, so it was recording my microphone in the VM. Yes, which I was surprised. I didn't think that would work. Me neither. Much good. So somebody said, silent or silent quiet reboot feature. What's that? I don't know. I guess that's the, is that the one with the grub hidden menu? Like if you, if you just reboot now, like it's not going to show you the menu period. So I guess that's what that means. Like if you reboot the VM now, it'll just go down and then come back up and it will never prompt you for the menu. Cool. Also, Marine really did a good job with this background. It is fantastic. It does look good. It's got kind of a, it's got a very fantasy vibe to it. I love it. It's been a long time since we've had a Fedora background with a fantasy vibe. The last time I remember personally was Fedora eight with werewolf. Was that Carmen? I forget which one like we, oh, I think Fedora eight was werewolf. Is there anything we could do to hammer on pipe wire? Not really unless you want to like hurt yourself by hearing audio while you're doing this. I don't want to try and do a loopback over a networked VM. Well, I'm also streaming and hearing like a loopback of the call. And I've like, yeah. No, well, I can test that later. Bye, Gerald. I've talked to you so many times and I've never actually tried to say your name. That's internet for you. Now what else are you going to do? I'm going to go on these cloud VMs and update. Oh, come on. Yeah, Neil. Your messages just came in now. Matrix server. No good. Well, that's why you got to be on the matrix or server. Or when the Fedora one is alive, that one. Oh yeah. Moving to that one as soon as that happens. Okay. Updated polling in the plugin. Switching it out. Release over. Oh, I wonder if there's any difference for the test. DNF current or DNF previous presently 32. Is there the instructions look about the same? It should be the same. You're doing the same upgrade procedure. Yep. They're the same. I don't know why they're different pages. Look at that. So much more replacing and stuff. So much text. We definitely need to do this more often. Live streaming is fun. Well, you can do this as often as you want, I guess. I don't know. If I'm going to do it too much, I probably shouldn't take up the Fedora channel every time. With this, it's like kind of, it's definitely strictly Fedora related on a Fedora day thing. So, but if it isn't like that, I would need to move to my own YouTube. Okay. The update to is done. Now we need to enable the debug shell and system upgrade reboot. These VPSs are moving so much faster than the VM. I wonder why that is. Oh, you're right. Now I've lost connection to the servers. That's the problem. Well, sort of. I'm going to set up some pings to the IPs of the VPSs to see if they're to hopefully see when they come up. But in the meantime, I need to see if I can get KVM video off the VPSs. I think I should be able to. Hey, there we go. Okay, let's throw these up on stream. Hello, Micah. Hi. You believe... Man, did I really just say her name wrong? It's hard. The letters. They don't match any sound things I understand. I'll just call her Duffy. That's easier. Hey, VPSs rebooted and back up. Well, you should take that long to upgrade. These were actually super fast. So something a lot of people don't know. The Fedora mirror service allows individuals to register mirrors that are public or private. And in either scenario, you can actually tell the mirror service what IP blocks it should respond to. So if, for example, a cloud service provider provides a Fedora mirror, they can also tell the mirror manager service that for any Fedora instances operating in these IP blocks that are owned by them that their machines will be provisioned on, the mirror manager will redirect them to use internal mirrors automatically. This allows them to fetch content super quickly and be able to have a good quality of experience when doing software updates. And both of the VPSs have successfully updated. Oh, I hate allergies. But now they're both back online. Nice. I can't think of any... It's not as easy to hammer these since they're the cloud edition and they don't have stuff you can open up. But that was impressive. It was just done. That is a testament to everything being a high quality flash and also having really close access to a mirror. Fetching quickly and then installing even more quickly. Is there anything else we want to test? Well, ideally I guess we would have something that... Well, since Fedora cloud of Fedora server essentially permutations the same test, I guess the only other one would be Fedora KDE. Oh, actually there is one more thing. Oh. I think... I think I set up an encrypted workstation VM. Somebody in chat says system CTL status. You also need to test GNOME software upgrades. Oh yeah, I do need to... I'll set up another VM because I forgot to take snapshots. Tisk, tisk, Grayson. Hey, I set this... I only thought of this like... this... I don't know, morning. I've had one day to get this all set up. I know. Still, tisk. You're so mean. Why don't you run this stream? Because I'm not pro. That's the point. I'm not seeing... look. Failed zero units. And over here, on the other server, failed zero units. Glorious. That's awesome. Let's go Fedora. Oh, where's the thing? Oh, wrong screen. See, I'm not that pro. You're very pro. I'm afraid that my VM might not be working. Oh, no. Finish installation. Okay, so I think this is an encrypted installation of Fedora 33 Workstation, so... let's do that test. You want to snapshot this machine before you do that? I guess. Thanks, Ellie. It was really fun. Nice to see you in the chat. I hope you have a good day. Good rest of your day. Thanks for coming, Ollie. Guest is not running. So, let's make a copy. Clone the disk. There's a snapshot button in Burt Manager. Just use that. Again, why don't you run the string? If you click on the little thing right next to the power button in the VM window, that's the snapshot control button. The power button. This one right here? You know where you see the play, pause, and then there's the one that looks like the power button? Oh, yeah. That button right there? That's snapshots. Oh, and I can create a new one. Well, it's too late now. I'm cloning the VM. Oh, jeez. All right, fine. Do it that way. In the future, you can just do snapshots from Burt Manager right here. In the future. Total. Question for the chat. How is my background noise been? It's a little noisy in my house today. This is a fairly good mic. And I've got a gate all on and all that. So, this is taking forever to create a clone. I should have just done a snapshot. I told you. You did, but I can't stop it now. Test, test. So anyone else in the chat who's been updating to updating along? There is a page. Why haven't I done that yet? There's a page where you're supposed to enter your results. So, I'll save that for when I'm done. But you click on the enter results for each one you've done, I guess. And you've got Workstation here and then Server, KDE, Spins. So, there's the Cloud. I'm not seeing any Xs. Oh wait, there's one. System doesn't boot. Some sort of Lux issue. There's another one. Overall, this page looks pretty good. Okay, let's go ahead boot up the Encrypted Workstation. Okay, so we're going through the encryption. You should do this one with Nome Software. Should I? It's more fun that way. Welcome to Fedora 34. Set up next. Skip. Well, we're just going to do Fedora 33, so... Aw, they hope I love Fedora. It's true, I do. Okay, so this one would be DNF No, GNOME Software Current Workstation. Is there no GNOME Software Current Workstation Encrypted? Wow, we don't actually have a test case for it. I guess it's kind of implied, because if you're able to do with or without encryption with regular DNF, you don't really need to beat that dead horse for GNOME Software because it winds up being the same code path. So you think I should just go ahead and do it through this? Yeah, do it through software. Software is more fun. But you still should probably apply software updates again through software first before you start. The difference is there'll be more reboots in your future. Okay, set up. Perform an installation of Fedora 33 with default partitioning. Do a full system update and reboot, so that's what we're waiting for to log the finished loading so we can do that. Software catalog is being downloaded. I am sorry, I am going to need to run for just a second. I'll be back in just a bit. And I'm back. So we need to download all of our updates. Yeah, I agree, Eric. I need like a status bar or something to show me where I am. All I've got is this cancel button and a list of updates. It's a little confusing. I wish there was more feedback in these things. I'm curious to see if when we're in Fedora Linux 34 there'll be more feedback in the GNOME software when updates are happening. So just a side note, it appears that my GUI has frozen. What? Why is it frozen? That's not good. For some reason it got paused. I must have hit the wrong button or something. Resume, please. That would certainly do it. It is not resuming. Why is it doing this? Maybe you were right about my server crashing. Just kidding, the server is still online. For some reason that VM has just decided to die. Well then Hakuna Matata. What? It seems a little random. Little. It means no worries. Well this is very much worries. For the rest of your days we'll get demonetized. Yeah, I need to stop. Well actually it doesn't really matter if we get demonetized. It's not like we're trying to make money off of this. Fair, but still doesn't mean you should set a bad example. Is it the VM resuming? This is really unfortunate. Got someone from Portugal. Nice to see you. Someone from the Ivory Coast. Hi. Cool. What does the SystemD Journal say about what does LibVirt say? Like what is going on? I'm going to switch back to my green profile and SSH into the box. I'm running these VMs on. Password. Pseudo SystemCTL statusLiberD What's that password? Oh, that's not good. Red text. No wait, when was that? 23. That was 24 hour time. That wasn't recently though. File from AgentMonitor Quimamo. I have a password manager Ryan. I was just trying to find the password. So the weird thing is that my other VMs are still working properly. It's just the one VM that is being weird. Is it possible that the VM could lock up and then display as paused in the in manager? I don't want to but I could always do the forced reboot. I can't get that VM to do anything. Force shutting it down isn't even working. This is really weird. I can't get this thing to work. Super weird. It's probably an... I'm assuming it's an error on my server since I can't make it do anything. And it appears that Neil has left. I'm still doing the juggling thing so that's why I'm a little in and out. Bye, Sev. So I'm just going to fire up the clone we made. You should probably... So you could try killing the process from htop or whatever. Kill-9 that Quimamo system x86 whatever and see if that makes it go away. I could. Trying to find the process at VM. Because I don't remember the syntax for how those processes are named. Oh, okay. This is not... This is very weird. My other VM has locked up too. You probably have a... That sounds like there's something wrong with yours. With your... What is going on in the system dejournal? For which? The server? The hostOS? The hostOS. So the server that you're running this on. Um... So let's do a... No, wait. JournalCTL? Yep. JournalCTL Dash... DashR R to see the reverse chronological order. Nothing important happening. On the server? That is running the VMs? Nope. Alright. What is... JournalCTL Dash RU Space... What's the server? LibvertD.Service? RU? RU. Reverse chronological per unit. Uh... Uh... And then LibvertD... LibvertD.Service. That's the right name for the door system runs. We really do need to, you know, make that server of the door server. I do, but that is going to be such a pain in the butt. Like... We've talked about this before. It's like... It's got so much running on it. It'll be... It is going to be so hard. So... For the JournalCTL RU LibvertD.Service Most recent stuff is just it setting up some name servers doing some DHCP DNS And then... Oh, it's in UTC That's why it's so weird. Okay, so you just need to subtract 4. So, about 20 minutes ago it had some sort of internal error, but 20 minutes ago we had the other VM running. Yeah, other than that, nothing. Yeah, but the other VM froze 20 minutes ago. No, I'm talking about the unencrypted one. Okay. What is the data storage underneath for the... What is the storage for the VMs? Is it LVMXFS? Is it ZFS? Is it ButterFS? What is it? So the VMs are running on QCOW2s sitting on ZFS. Alright, what does ZFS say? You could have bad disks and your storage could have locked your I.O. All disks online. No known data errors. So Z-Pool status and it shows everything's happy? Dash V for verbosity. And data errors. Another thing is that the normal workstation install that we updated fine is still running and it's not having any issues. So it's something wrong with the encrypted install. I don't know if it has to do with it being encrypted or if it's something broke while it was installing. Well then... Yeah, I think it's definitely... Well at least you have a reproducible test case. You get to file a bug report. Why do I file a bug again just though? I don't know. Just because it's in a VM and it's pausing itself and I can't unpause it. So is it something wrong with the VM software? Is it something wrong with? Bye, Ryan. Thanks for coming. What does verse status say for it? Verse? Or VertManager. What does VertManager say? What is the status of the VMs? I assume that the UI, the VertManager UI is not frozen. It's the individual VMs are paused. But yeah, you can see... I can't see anything. You're not updating anything on the stream. That's true. I'm not. Yeah, I don't know. I didn't feel comfortable sticking my server's journal CTO in. That's fine. I would probably say... Yeah, I have no idea what's happening. I would probably say this is... Yeah, I don't know. I don't know what to do here. Encryption is probably... Yeah. So like, usually, as Josh said, usually when a VM gets paused, that means something has happened on the LibBert side that says that it needs to pause it. There's a couple of reasons that it could be caused. It could think that there's a CPU exception, so it'll freeze the whole VM and stop it. So that would mean that your CPU has bugs in it, and then they were getting tripped by things that the VM was trying to do. Another thing that could happen is that it hit some kind of error state managing so that would cause it to break as well. But I don't know. You need to ask LibBert to find out what actually happened. Does LibBert have a log? Yes. I don't understand if... Would it not be showing up in just a system CTL status LibBert or a journal CTL? It might or it might not. It depends. I would expect it to show up in the journal, but knowing what I know about your server it's entirely possible that the variant of LibBert on there is probably not configured to do proper logging. I was making on a happy face at you and then I realized that the stream log made it useless. Yeah, I'm trying very hard not to say what's wrong with your server. Which is everything. Stop. You're over... This is ridiculous. Don't believe him, stream. Well, at this point I think we can call this an end because I don't think we can do anything else. Oh yeah, Pervita logs are in Varlog LibBert... No, they're not. No, they're not. Because... No, yours it'll be fine, Josh. The problem is his computer is made of terribleness. No, it's not. Hold up, hold up. Let me find where the logs are actually stored. They might actually be stored in that place, but I would be shocked if they actually left that alone. In Varlog... In Varlog... Varlog LibBert QAMU. That's a directory. Oh cool, they left it alone. Yeah, go ahead and take a look in there. There should be logs about each individual VM. Sorry, I don't have anything to put up on stream right now. You just have to see my face. Which isn't great. Dome... Oh, in there it's just got a key and socket. I don't see any logs. This might be on stream related content. I don't know. Is there anything else we should test for the stream? The encryption might just be something I have to figure out tomorrow. Well, you could do an unencrypted... If you want to install Workstation again from scratch, you could do unencrypted and do Dome software. Because I didn't make any snapshots, yeah? Yeah. Okay, let's shut down good VMs. Uh-uh, get rid of these clouds instances. That's the only other thing worth doing. And then you might want to talk to Sumatro and the Fedora QAP people to help you figure out what's going wrong with the encrypted ones. Because man, that's weird. And I have never seen that before. Okay, let's make a new virtual machine. I mean, yeah, I don't know. Okay, with Fedora 33, 4GB of RAM. I am super impressed that we still have over 20 people watching. So happy that people are enjoying this. That's because they love you. You're doing great work, you're pro, and you're amazing. Mmm, meh. Wow. Thank you. Wait, wait, what? Oh no. No, wait, what? Why did it stop counting? So on the with the new VM, it's doing the ISO check and it just stopped at 31%. Oh wait, it's moving again. 32. Why are you doing the ISO check? That's a good point. I've used this ISO through times already. We're good. It's the escape button. We're good. Escapio. I don't know why this is even still here. Cause you gotta check that ISO. Make sure you didn't miss anything on the download. Is that not a thing? No, not in practice. I always do it cause it says to. The thing is, if it were actually corrupted for some reason, you'd be suffering way worse problems long before you would be able to install. Thanks, Robin. Oh, you're right. Let's check out what the memory. Oh, yeah. No, that makes sense. Then why did the other VM keep running? Why did it kill those? Because there was more disk IO. Cause it's an encrypted disk. So that means that it's an encrypted thing on top of that. You basically have the worst case for the Android file system. Oh, yeah. We've actually got a good bit of memory pressure going on here. That's probably what killed it. Because when you're doing encrypted disks, that means that all the IO is totally random. Which makes it the worst possible case for oh, yeah. No, I've seen this before. Now that you have the different pressure bit, I understand what's happening. Yeah. This is because you're using it on ZFS. Hmm. Yeah, we've got 7.22 gigabytes of 7.68 gigabytes used. You do not have room for more VMs. And swap is about that type, is about that type too. You definitely don't have room for more VMs. And that's why we're making another one. You're also turning off the other one because you don't need it running anymore. I know. It's also unfortunate that this is the same server that's running my DNS so it might be bad for the stream if I started eating everything. Welcome to my life today. Is that your life? Right now it is. Everything's on fire and DNS is not working. Are you still fighting with the GitLab issues? Oh no, that's fine. Everything's fine there. This is just, this is totally different. Let's see. Memory is being eaten up by virtual machines. And then my SQL. I didn't want to run the VMs on my machine while I was streaming because I know this machine is like super pro and all, but I don't know if it can handle that much. I think you need to build a new server. Is that just so I could install Fedora on it? No, it's because you actually don't have enough capacity to do all the things you're doing on the one that you have now. You're paying for it? No. Exactly. Why don't you pay for it? I don't have any money. I've got like a grand total of 30 bucks right now. Well, that 30 bucks could buy your something. You could buy your Raspberry Pi. I'm just gonna get one Raspberry Pi. No, that'd be a serious downgrade. Configuring story. Why is this VM? This one feels frozen too. Oh, did this one lock up too? Yeah, this one feels locked up. It doesn't say it's paused, but it's not responding CPU activity is that none and it hasn't. Oh my gosh. Why are all my VMs dying? What is the IOS starvation? IOS starvation. That's what's happening there. Oh, my disc light isn't even lighting up yet. My memory is just dead. Okay, so you've killed your server. Good. It's not like I don't use it for everything every day. Right, so that's probably a good point to end the stream then because your server is basically ruined and I don't think you can you can do anything. In that case, thank you all for coming. This has been super fun. Definitely need to do this more often. We'll probably do this again to do the rest of the upgrades. Exactly. Oh, I still do need to file my results. Yes. Should we leave the stream running for that? Yeah, why don't you go ahead and do it in the stream? That way people can see how you actually file results. Yeah, you can watch me figure out how to do it too. Never mind, I actually found this before when we switched to ButterFM. Yes, you did. I made sure you did it and that's why you have the buttery badge. Oh yeah, I remember why I did it because you bribed me. Okay, so let's start with DNF Current for Workstation. Did that. So, username is PeterKid profile. What's profile? Is that just me again? Yeah, I think so. Where's profile? Below user. Oh, oops. I can't see that part. Yeah, it got chopped off. So I'm just going to put down as it's pass and it works perfectly for Workstation. Submit query. So now Oh, profile was for my device. Okay, we'll just put that as KVM UEMU. I don't know if it'll edit it or if it will. That's not helpful. So, now you see me fail. I'm sorry. Profile is apparently supposed to be your device type. So I've got that done. Well, you can fix it in post. Just do it again. I did. So now we also did the cloud spin. So we did a DNF Upgrade Current. So that was me on KVMU Cloud Spin in Digital Ocean Result Past Work Perfectly Submit and then another one for Previous because I did that too. And then that should be all Anyways, now I really should leave because it's dinner time. Yep, sounds good. Thanks everyone for coming. Thanks Neil for hanging out. Answer all the questions that I don't know. I mean, it's fine. Do this again and have some more people. Definitely. And now I'm going to hit stop streaming. Goodbye.