 Welcome to the homelab show episode 102 red hat and why open source is hard and we're going to be discussing the red hat recent things here in June of 2023 changes they made. And there's some challenges and open source that I don't have all the answers to but me and Jay want to help bring them up so people can think about them. That's our goals just to inform you here not here to provide the ultimate this is how all open source things to be run because unfortunately, I don't think that answer we've really figured out as a community yet. I still think open source is the right way to do things, but there's some challenges that come with running these large scale projects and yeah, and by the way, we can at least agree on one thing we don't think red hat did the right thing. No, there's a lot of face palm regarding this but there's also a you know industry thing at the center of this as well and it's a. I mean open source has been around for a long time but even though it has it's still kind of new it's a new idea when it comes to the human species it's like, you always want to hold back everything because we have the cutting edge stuff and you know we don't want to give anyone the blueprints but then once people realize that openness is better. It's a long transition and maybe some people are still kind of in the past on it but we'll talk about it. Yeah, we'll be talking about that. First, let's jump over to some feedback because there's a couple of questions people had and I think it's worth revisiting and the first discussion is going to be on. Let's talk about syncing things with syncing or some of the other services that are out there and that I think this is where people get a little confused on this, and there's different use cases for each one of them. And essentially, when it comes to syncing, I like real time syncing, which is what syncing does syncing is going to say hey, all your data lives on each device and then you sync it between all those devices in real time. Or when those devices come back alive. I like this better than even backups and because of syncing having revisions and I keep snapshots and things like that on my true NAS where syncing runs. If for some reason in a device was attacked, I could always roll back to a previous state, which is easy enough to do. But it's also nice for things like what Jay uses it for, which is to me one of the really clever use cases that you use syncing to sync all of your retro stuff. So all the save game states are in the same place at the same time. And syncing is multi-platform. It runs on your Linux and Mac and Windows devices. And this makes it a really, to me, a great choice for real time file syncing. But you can and NextCloud has a file syncing utility as well. So you could facilitate it through NextCloud, but NextCloud has so many other features, and you can actually edit the documents in NextCloud and synchronize them. That's a different type of solution where it's more of a fuller solution. But I don't like the complexity of it. That's why I don't run that. But of course, the good old tried and true using a share. The good and bad about using a share is the files aren't managed in real time on each device. But that also frees up space on device. But that means each device has to connect to that share. And that can be challenging if I take a laptop. And I've done a video on this. There's a reason SMB does not run well over VPN. It was not a protocol designed for that. So file shares over VPN are less good. So syncing with syncing is still one of my go-to choices I have for real time syncing if that's your use case. What's your thoughts, Jay? So a couple of things. NextCloud, when it comes to syncing, is great. And I've used it for a very long time. I still use NextCloud, but I don't use it for syncing anymore. Nothing against NextCloud. It's just, I like syncing a little bit better. But taking that out of the equation, syncing or NextCloud? So the bigger question is, do you have more use of NextCloud than just syncing? If syncing is the only thing you want and you are not interested in any of the features of NextCloud, it's overkill. It's bloated for that. It's not bloated in general, but it's bloated for that. You're installing NextCloud as a syncing solution, but you're getting the kitchen sync with it. But with, when it comes to remote access, that checks the box as well, because you can access your files through NextCloud on your phone and everything like that. So that does check all the boxes for our listener here. But I'll just say that sync thing I really do love, it's, I think it's just for the same reasons you mentioned why you love it. I do a star topology with sync thing. And I feel like this works very well to where I won't use it any other way. Some people will have every device syncing to every other device. I think that's just craziness. You know, in my opinion, you have syncing and TCP traffic going in every which direction. I don't really think that's the best way to do it. But with a star topology, I mean, you're only one person. It's not like you could simultaneously use two computers. You could have something running on one computer and something running running on another, but your hands can only touch one keyboard. So the idea is you have a NAS or a central server could be a syncing server, could just be true NAS, could be anything. Disk station from Synology, for example, doesn't really matter what it is. You could have syncing on that device. And then every device syncs to it. They don't sync to each other. They sync to the central device. That's why it's a star topology. And that central device becomes like a central source of truth for your files. And in my case, true NAS is where the syncing hub, I'll call it that, but it's not. That's where it is. And since true NAS has revisions as snapshots, I have revisions in syncing. I also have revisions in true NAS as well. I have revisions of revisions. I even have backups of my previous files that go beyond what syncing even goes back to. If I really want to go back in time. And that system is just made so easy with syncing. But when it comes to our listener, though, wanting remote access, especially considering VPN and Samba are the best of friends, I think maybe in this case, NextCloud is the best fit because it does get that access externally. So there you go. Next is different options for me. Like I take my laptop and I travel with it and I like having all the usual little bash files and scripts. That's one of the things I sync with my syncing, all my little things I need. And if I change any of those things, as soon as it comes back into contact with the rest of my network via VPN even syncing, we're sure VPN it's in sync. We actually share this between a couple of employees because we have different documents that we do there too. So it does contact resolution and things like that, but either one of them are good solutions syncing or NextCloud. Yep. So definitely recommend that. The next one's easy to answer and is going to be controversial is someone wanting to build an all IPv6 home lab. And I wish you the best of luck, but you're going to find that as much as you, it is a learning opportunity. I know that IPv6 has been the next big thing next year for a lot of, a lot of years. There's even a joke we did at one of the talks at one of the Linux users group or computer users group talks, how IPv6 is the next thing and a person kept doing the same talk for like 12 years in a row because it kept getting up on here. Every time I did it that next year is IPv6. The reality is I don't know when or if, but I actually in someone can call me out on this. I don't think we'll ever really get to IPv6. I think something will come up before IPv6 that we adopt that solve things differently. I don't know what that thing is, but I don't think in the future we go to IPv6 because Nat. And by the way, I think I tweeted this out. You can find it's a great mini documentary. If you want to call it on YouTube called the history of the Cisco pics firewall. And it's the people who invented Nat. It's really neat, good video, great information, a lot of history. And once Nat solved that problem, whether you like it or not, it is the solution that has been pushed out. That means IPv6 is not it solved it so you don't need IPv6. I know there's some people, but I want a fully public IP space, blah, blah, blah. I'll also throw out the unpopular opinion, but it's backed up with some stats that you can reduce your threat service by turning off IPv6 because there's been a couple attacks that involve the IPv6 stack because it's not as well tested. For companies implementation of it. So sometimes flaws have been found in it. So yeah, that's another issue. And there's actually, I wish I could find the write up on this to link it. But if you Google it, there was a, I can't remember if it was in dark net diaries or one of the security podcasts listen to. They could not figure out why or how I should say exfiltration of data was occurring. The threat actors were using IPv6 because it bypassed firewall rules and their net flow of stuff that they monitored traffic with couldn't monitor IPv6 the way it was configured. So they actually when they brought in a third party security people, they're like, yeah, you're right. The bandwidth is definitely blinking. It's lots of bandwidth. We don't know where it's going. Oh, we can't see the data because they only the threat actors are only using your IPv6 out of your network and your your tracking tools weren't enabled for IPv6. So yeah, turning it all off is still my thing to do. I know that's going to be an obvious opinion, but hey, leave your comments down below on that. I'll take the hate in the comments. Well, I'll back you on this and you know, I'm just going to ask everyone to keep an open mind. But before I give you my opinion on this, one thing I'll start with is Matthew or listener. He has a very good reason for implementing IPv6. He wants to learn it. And that's important to keep in mind because if you want to learn something, it doesn't matter what I think it doesn't matter what my opinion is. If you want to learn it, you should learn it. It doesn't matter if I like it. And even if, you know, someone themselves also shares my opinion, their company might be mandating that they learn this and they might might not have a choice regardless of how they feel. So I always try to keep that in mind when it comes to HomeLab. IPv6 is never going to happen. Don't waste your time otherwise. And I know that's also going to be controversial, but let me explain why. I think I first started learning IPv6 around 2010, 2011. I can't remember now. And around that same time, Carrier Grade Nat was announced. I'll get to that in a minute. But since I started learning IPv6, let's put this into perspective. Okay, back in 2010, 2011 or so, probably the first time I ever heard of people saying IPv6 is going to take over. Since then, I have gained nine IT certifications. I have finished a master's degree. I have worked my way up to director of operations at the company that I was working for. And then I built a platform on YouTube and started my own company in that same amount of time. And we're no closer to it being adopted. That's just fact, okay? The other side of this is I had to Google this because I can't remember this number. Okay, so if you don't have 340 trillion, trillion, trillion devices on your HomeLab, you do not need IPv6, period. You just don't. It's going to be problematic. IPv6, you know, has all kinds of flaws, but not only that, it's for ISPs. They're the only entity on the planet that can saturate or even put a dent in the number of IPs that are available. So I really don't understand like the whole IPv6 at home thing. You're never going to hit that number. You're fine. ISPs are the only ones that will saturate IPv4 addresses inside your local network. You know, behind that, you can have as many, you can have the entire IPv4 address space if you could find a way around the RFC thing. But in reality, you know, he wants to learn it. He can learn it. But we have carrier grade NAT, which I first heard around, I don't know, 2011, 2012. I first heard of that. And I think it was Verizon that was in the news at the time. And as soon as carrier grade NAT was announced, I'm like, that just killed IPv6. It'll never get adoption now because that's the easy mode. That's if you give someone like anyone like, and I'm not talking about enthusiasts like us, I'm talking about your average IT department. Implement IPv6 or continue to use IPv4 but with carrier grade NAT. They're going to go with IPv4 because it's less work. They're already overwhelmed. So the situation with IPv6 is just, it's just baffling to me that people think it'll catch on. And this is nothing against our listener, by the way. Yeah. Our listener has every right to learn it. And I totally encourage them to do so. He wants to learn it. It doesn't matter what I think. You should definitely learn it. I'm no longer talking about our listener, just IPv6 in general and the community in general, not that individual. It's just IPv6 is just like that mean girls meme. I just see Regina George saying, stop trying to make IPv6 happen. It's not going to happen. So I guess I'm the one that's going to get the hate now. You're safe because I didn't want to rant about it. But I mean, this is a rant where I'm looking at like over 10 years of paying attention to this. We're no closer now. We're not going to get closer. So ISPs are going to enjoy it. And here's the fact, everyone that's listening is probably using IPv6 right now, almost everyone, because if you check your cable modems IP, you probably have an IPv6 address on it. So most of us are using it, but that's the right place for it to be at your cable modem. After that, there's just no more reason for it. Yeah. You'll run into so many. It was evenly recently that Shurnass allows for you to assign. I believe they have an option now and I've never tested it that you can assign IPv6 only to it. It used to be something you couldn't even do. So it's slowly, but look how long that project's been around for them to get that part going. So yeah, it's just not it. I don't, I mean, try it. It's great. The learning experience and the more you learn about how the stack works, it's an interesting solution. And learning about all these things is actually super interesting. I love some of the history of how we got from where we are or from where we were to where we are with this, but it's, you know, I don't see it as a viable option yet. I think everyone should learn the basics of it just to kind of understand the basics. I think that's that's fair because you're going to see IPv6 addresses. Maybe it's not what IP you use to get into your, you know, your proxmox device or something, but you'll at least see it at the cable modem. So learning the basics is fine. It's just as much as I love new technology, let's face it. We haven't even moved to Wayland yet in the Linux community. So there's all these things that are trying to become the new normal, but it just takes a long time. Yep, for sure. So that's all the listener feedback we have moving on to some software updates. I heard proxmox based on the number of people that keep messaging me on it has a new version. Well, so disclaimer, I haven't, I have upgraded to it. I haven't really dived into it because I've been busy migrating things to WN12. I did take a moment, I think it took me like an hour and it only took me an hour because I have VM disks that are on the storage that when you migrate those have to migrate the disk at the same time. I don't have shared storage. So I had to move the VMs off of one, upgrade it, move the VMs back to it, upgrade the other. It was pretty straightforward. They have, I think it's like PvE 7 to 8 command. So you update your current one and you can run PvE 7 to 8 to 8 and it'll tell you how it thinks your health is. If there's anything you should look at back up or what you should do to get that upgrade underway, which is cool. So I did that, did everything it said to do, upgraded to it and it was fine. And then I just had to get back to WN12 migrations because I had a lot of work to do, but you could probably argue I shouldn't be upgrading Proxmox in the middle of a work session because I'm just asking for trouble, but it's been fine. When I look at the release notes, and again I haven't went full in depth into this yet, the biggest thing is WN12 Bookworm is what Proxmox is now built on top of and there's a ginormous number of improvements. Nothing that I think anyone's going to brag about unless you're using a technology that I don't use where a feature might not apply to me, but I think they did great work on it and I noticed little tiny improvements everywhere. Like I think I was complaining when they went to dark mode that it was kind of fuzzy, blurry, the color scheme was okay, but it just kind of seemed like it was rushed, but now it looks crystal clear. It looks awesome. It doesn't look like it stands out as very different. It's just that on a higher DPI screen it looks fine, which is a great improvement. But I think the fact that I have not all that much to say about it is a good thing because I haven't had any problems and everything's been solid, so that's pretty cool. Yeah, I still haven't, maybe one day I'll try Proxmox, but it's really, really low on my priorities list, because I'm still a big XCPNG person, so that's why it always surprised me how many people asked me and DM'd me and tagged me going, what do you think of the new Proxmox? I said, I still don't use it. I don't have anything against it. I just don't have time to use two hypervisors right now. It's interesting how people, at least at the beginning of the podcast, it's always like team XCPNG, team Proxmox, which one are you in and why is your team better than the other team? And they would ask me, which one's better? And I'm like, I like both. It wasn't for the fact that Proxmox had containerization built in. I mean, that was literally the tiebreaker. I could have flipped a coin and went with either one and then just as happy. But I thought Lexi containers would be a nice bonus, and that was something I wanted to cover on the channel anyway. So I figured, I'm going to cover Lexi. Why not use the thing that has it built in? And that's why I made the choice. But they're both great. Yeah. Both solid. I, whichever one makes you happy, whichever one matches up with the features that you're looking for. So. Yep. All right. Now, I did link to in the show notes, your article you wrote, how Red Hat's open source negligence is doing actual harm to the Linux community. So I'll let you start probably by explaining first what Red Hat did for those that haven't taken the time to read through that and understand how they're changing things up a bit. Yep. So I'll keep this super short and simple because I know, you know, some people already know full well what's going on, but there's going to be some that, you know, are like fine, I'll listen to this podcast and learn about that Red Hat thing everybody's talking about. So if that person's you, then basically we have downstream recompiles of Red Hat. We always have sent to us famously is that, or at least was that scientific Linux, which doesn't exist now was that. And the idea being that, you know, Red Hat is making an enterprise Linux distribution, you get it via a support agreement. And as part of that you get the software and it's been that way for a long time. So Red Hat has had a way of selling support contracts, but, you know, also giving the source code away for free. It was kind of like this perfect thing there, or semi perfect anyway. So the idea being that when Red Hat releases a new version of Enterprise Linux, they, you know, have the source codes available so if you wanted to create Bob's custom Red Hat and or not Red Hat you can't use that name but it feels like Bob's custom Enterprise Linux you can totally do that until now. So, the interesting thing about this though is that sent to us being a downstream recompile of Enterprise Linux up until more recently. And Red Hat basically acquired it they use, you know, joined forces but we all know what that means some time ago, and they thought enough of a downstream recompile to own it. And, you know, everything's been fine since something and tell us not fast forward blah blah blah things happen to between up until more recently where sent to us stream is the sent to us that exists now so it's not a downstream recompile. Sent to us stream is the upstream of Red Hat Enterprise Linux now. So it's not the same thing anymore. And basically what ended up happening is red hat is like, yeah, we're not going to keep sent to us going like it was. But if you want to do your, you know, downstream recompile thing the source code is right there and it said it right there in the FAQ, which they've crossed out now. And it said right there in the FAQ when they switched to stream they literally said we're going to keep the source code available for those that want to do a recompile. Now, they're saying, yeah, no, we don't want to do that anymore, we don't want these downstream recompiles to get our source code, and they are making it very hard for them to exist. And all the Linux and Rocky Linux among others are the new downstream recompiles that are basically like what are we going to do and how do we get the source code if we're going to continue making this distribution. Red Hat is effectively, you know, claiming that they are, you know, all for open source while they're literally making it hard for open source projects to exist they're doing harm to other open source projects. It's just to protect theirs. And the situation has a lot of people angry because, you know, they said they're going to keep the source code available. Rocky and Alma were kind of depending on that, just for them to, you know, renege on that and take it away and say no thanks. That's the situation is that they could do whatever they want to be fair. That's the thing that we have to understand here. Red Hat owns a trademark. Can they do this? Yes. They could call it like ultimate Linux and have a different name for it. If they want to do they have the trademark they could do whatever they want. They could take the source code away there's a legal gray area won't get into but I think the problem here is that we're angry about something that Red Hat is fully allowed to do. What they're doing is more of a betrayal of everything they stood for in the past and that's kind of where we're at. It's an overly simplified explanation, but there's an article like Tom mentioned on my website you can read if you're more of a video watch or I have a video on this so however you like to consume information. I already have more information about this if you'd like to learn more but that's where we are today. And this is what we want to bring up open source is hard and it's a business model that does work but with some of these large companies being involved sometimes they are well not forthcoming or not doing things in the best interest of the community. And this is where I get it and I understand there are freeloaders or are people who simply grab a project that a lot of people put a lot of work into compile it slap their name on it and just make money with it. And that can be a short term gain and I'll throw a company out there that I've talked about that I know people are going to be angry because they don't think about the entire ramifications of it. We've been using PF sense for a long time and people like oh no Tom, didn't you know they get angry when people fork their code and did open sense and blah blah blah but there's reasons for that and I'm not justifying any of the bad behaviors they did but I want to explain the business model problems. Netgate employs a lot of people to support the PF sense project. Most all the code contributors to PF sense happen to be a neck gates payroll. There's a few more people on that gates payroll that are incredibly important to the free BST project. They actually support kernel developers that put things up there and the BST and how does neck gate pay for all this because it turns out you may want to write code for free but someone still has to pay your bills and it's not as simple. We don't have this current economic system where you write code for the greater good of the community. We shall bless you with a free living. Turns out that doesn't occur right now. Therefore someone's got to put you on payroll. How does that work? Well, that gate sells some hardware appliances and what if someone wanted to just go, I'm just going to compile it myself. I can undercut them because I have all this cool stuff in the kernel that then builds into PF sense that then allows us to sell things. They're still great. You can undercut it because you're not paying it. But eventually if you don't have a company neck gate and there's plenty of other companies like this that are doing all these kernel contributions to BST. What happens to BST? It goes stale. It gets old. It doesn't get new drivers. It doesn't get an updated kernel. The project doesn't move forward. There's a lot of businesses IX systems and our company I work with. IX systems matter of fact, they just announced the other day because they contribute greatly back to the ZFS code base. Same answer. They're selling hardware. That money funds a lot of people who write a lot of code that goes into ZFS, including some of the new rate expandability options of ZFS. That takes a lot of really heavy engineering and someone's got to pay for all those people to keep pushing this code out there. So open source really it may be given away for free, but the time it takes these people to write it is not free. Now, Red Hat and Jeff Kearling certainly pointed out because Red Hat, didn't they call people freeloaders? Was that the term they used? Essentially, they said it's disingenuous that people are according to them, downloading the source code and renaming it without adding additional value. They didn't mince words basically, but that's effectively what they were saying. Yeah. And that's where these challenges come in because these people are not in many times Red Hat's accepting a lot of community contribution and things like that too. You see some niches where the community contribution isn't what it used to be. It's just some small group of people, but it somehow has to pay and fund all these people. These are some of the challenges that come in. Even to pivot a little further off, if you read the debate between Elastic and Amazon. Amazon and Elastic Search is a pretty popular product, but Amazon says, hey, cool. Glad you guys developed that. And how does Elastic fund development? Well, they offer hosting services. How does Amazon make money? Well, specifically not the selling part of Amazon, but the Amazon Cloud Services. They sell hosting services. They started, as I understand it, undercutting Elastic because they're big. Like, look, we can offer this and take all your code and offer hosting. So Elastic changed your license to allow it to be open source. You can have the code, but they have a license that doesn't allow you to resell it from a hosted standpoint. Other companies are starting to adopt that as well. You still get the source code. So it is open source, but the licensing says, no, you can't grab our source code and offer it as a service. That's part of the terms of conditions. So we're starting to see different business models kind of fight against it because if they don't, if these companies don't exist, who writes the code? And that is where some of the, we come back full circle. Now, the Red Hat situation, they're not an unprofitable company. Matter of fact, they're doing very well profitability-wise. This is where you got to try and figure out where that dividing line is between doing well and hyper doing well in terms of are they just getting greedy? Well, that's kind of the market pressure will determine that. Not necessarily. And Tom and Jason don't determine that, but market pressures will. And this is where we're going to see what happens with Red Hat. So there's a couple comments I want to respond to here that I think is going to help form my opinion. Before I talk about the first one, the main point here that I think we're really trying to get across is this is like the Kobayashi Maru. If you're familiar with Star Trek, the unwinnable game, right? You can't win. You just have to lose as gracefully as you can. And that's how you're judged. An open source can sometimes be that way, right? Or anything could be that way for the most part. There's downsides, you know, there's no such thing as something that's all 100% benefit. Okay, keeping that in mind that we're not going to come to a definitive answer here. We're basically putting questions in your head to help you. Our listeners come up with an opinion if you don't already have one. But first of all, the comment is open source isn't a freeware. And that's true, right? Just because it's just because something is open source, that means you get the source code that doesn't mean that you're getting it for free. It often is the case. I mean, you can go on GitHub and there's a ginormous number of open source projects there, but it doesn't necessarily mean that it's free. And that is absolutely true. So I wanted to bring that up and just acknowledge that that is the case. Now, the second comment I wanted to bring up is by a friend of ours Veronica. Veronica explains where she's basically saying or the comments right there on the screen and for watching the video version. Well should not block access to the source code even if it's just a development account. It's a thumb of the nose to the community that builds them up now. Let me let me there's there's a couple aspects of this I want to I want to mention first. I personally don't feel that if when red hat enterprise Linux first originated when it was invented okay when first came out. If red hat had this system back then up until now I don't think anyone has would complain because it would always be that way red hat could do what they want. Yes, but the other side of this is that when you put anything on the internet. It's it's eternal. You don't put something on the internet and then think you're going to take it away. I could have a spelling error and a blog and somebody could make fun of me for it. And I could fix it but if somebody screenshots it or the go back to the way back machine you can I know it's a stupid example but the point remains. You know if you put something on the internet it's eternal and it's what people generally expect now with red hat. They made a business model around fully open open source fully available full collaboration was all over their branding it was all over every part of their company. And that was what red hat did okay and that's kind of the most important thing here to keep in mind is that when you go one direction and go another. You know that puts people in a bind now to be fair companies go different directions all the time. It just happens but the issues that I have here with this in in just to kind of put my criticisms on the table here is that. Red hat saying one thing and when red hat says something enterprises listen okay keep in mind 10 years in enterprise is common for something to be in production. So when an enterprise company makes a decision, especially when a company like red hat is an enterprise it company they have enterprise all over the website go ahead and look you'll see it right there. And when a company uses a vendor, they are trusting the vendor at their word red hat said for example we would get 10 years of support with sent to us eight they stripped that away. In my opinion that should have been the end of red hat, you know, that's taking taking something away from enterprises where companies all over the world decided to implement sent to us eight, because they were promised 10 years of support. And they had no reason to think they wouldn't get it because they always got what red hat gave them for support before that red hat abruptly takes it away just a year, just over a year later. At that point you can't trust a vendor that's going to take away the support they promised you. Yes, sent to us is free. And you could argue that you know they're not paying for it so what's the problem but it's not about that red hat said they were going to give it give that away for free and provide that and they decided not to do that. They also said they're going to keep the source code available, and they decided not to do that. So, it's less about can red hat do this is it okay from a legal standpoint yes it's okay from a legal standpoint yes red hat can do this they own the trademark. Should they do this is the problem. And there's no business model that where you make your customers angry that's in your benefit right now. I hope Veronica doesn't mind me mentioning this but you know when they're chatting she brought up a client moving to Debian and because of this, and it's like, okay. That's very telling because red hat is thinking if they put everything behind a paywall. Oh people will be forced to buy rel no people are probably going to get tired of red hat saying one thing and doing another and go to a completely different platform yes it's orders and that magnitude more work, but that's we're talking about enterprises here they have to trust their operating system regardless of if giving it away for free was a bad idea that's still a red hat said that they were going to do now, getting back to the whole thing about open source is hard and taking my opinion out of the equation. When you release something as a company. And you put it out there and open source the open source software you're accepting all the risks, but that comes with that your code is going to be looked at. It's going to be downloaded somebody can fork it somebody might be able to sell support services around it. Somebody might change it recompile it and make it into their own thing and profit from it. So, is that kind of, you know, a bad potentially bad situation. Yes. But when you open source something you're taking that risk that somebody might do that. That's just the way it is. Now if you put out open source code behind a paywall. Obviously that can't happen you don't have to worry about people creating a spin off or recompile because you're literally making it illegal for that to happen. But if you don't do that and you make it fully open to the point where literally everyone is able to get everything in the kitchen sink to make it, then taking it back I mean people are going to fork it and that's just the way it goes. So, if you open source something understand the risks the other risk is that well maybe not necessarily risk but a mindset thing open source is not more secure. It's not, it can be more secure, because open source code can be automated or automated well can audit it. But it doesn't mean that somebody's going to take the time to audit the code it's an enormous amount of work and costly and all these other things. But the difference is whether somebody can audit it or not I just rather have a situation where someone can audit, and I hope that they do, rather than choose something that I have no idea what's going on with the code. I think that's why we choose open source because we have to have a paper trail we have an audit we have a commit list we could follow we can see what people what's going on here. We could trust it more because we know it's going on with it but it's not inherently free like what was mentioned, and it's not inherently more secure. It's a, it's a software model. That's what it is and that's all it is. Yeah, it's really. It is a challenge because you kind of mentioned just because it's open it's not secure, but also let's use bit warden there's another company that we have both talked about there are great open source company I love their tool. But someone has to pay for auditing all that code and bit warden took that task matter of fact they regularly do that task. That's why they have to have the business model they do around there because I have commented before people are like oh Tom why don't you even use bit warden they have license fees and blah blah blah you could just use an alternative back end and I'm like because I want to support the project I'd like to see this project flourish. If everyone just started using alternative license free back end that doesn't contribute code back, which is the bit warden rust implementation and if you just want to use it because you for simplicity reasons it's fine I'm not telling you not to it is open. But now you can see how that affects their business model if more people started doing it or everyone starts just pushing to the unlicensed version. Right you have a problem or let's go back to invoice ninja I use invoice ninja their licensing fees are like $30 a year. That's my contribution back to the code they make money by hosting. But there's actually people who wrote scripts to remove the $20 year fee. All it does if you it gives you a little license key but by the way because it's open source you can see the source code and delete that section of the code that says you haven't bought a license. I've seen people like posting pull requests for that I'm like really really you don't want to give us $20 a year. They're like it's not open source if it has an error if it has a message that I'm using an unlicensed version I'm like really. That seems so at some point these companies need some way to recoup the time they put into auditing the code, securing it fixing bugs and there has to be business models around it like you said we wanted to raise some awareness is a lot of what this is. And Red Hat said easy target but Red Hat's also charging these enterprise companies for all these support contracts. So they're doing the right thing. I mean Red Hat when they were bought by IBM was in the realm of about $30 billion company for their purchase. They're doing all right. So I feel like they're on the side agreed. I think there's more there's way more to the Red Hat story and here's the thing. I know there is, because I've heard it from the people directly I can't say anything that I was told because I'm not that kind of person that someone's going to tell me something in confidence and I'm just going to blab it out I've had people tell me some of the things that are going on there. I think the picture is very clear what's going on in my mind when you when you put the connect the dots and everything but you know here we are a situation where I got to kind of keep that, you know I have to stop that there but what one thing I'll say though is that with it's just ironic red hat is saying, you know, we want people to give us money and they should, but they were floated and, you know, created from on the backs of people volunteering that probably won't see a dime. It's just ironic, but the situation being what it is it's you know open source is very difficult and I don't mind paying for software. I don't think anyone should I here's what I think everyone should take back from this, at least give back and I'm not saying you should just learn to be a kernel developer or something that's hard. That's not what you what you're good at or not what you're passionate about. You could give you could, you know, submit bugs you could create documentation but I think the bigger thing here, especially for home lab is some of the software we use in our home lab kind of graduates and becomes what we use at work, because you know everyone everyone in the home lab that works my tea, you're just waiting for that moment, where their boss gets everyone in the room is like okay what about this technology where I think about having us use this is that you guys know about this and then you're like yes I run that, like every day for the last five years. That's that's what we hope for. Now, if a company that you work for switches to this open source technology and they download it for free. And that you encourage your company to donate money to the project if they're going to save. I don't know, like, I'll just make a random number $20,000 and licensing fees that they don't have to pay because they went to an open source technology, you're saving that money. So if you're saving that money, put some of that money or just point that money toward the project you're still saving money but you're also helping the open source project that helped you save money. Yeah, I think it's fair company should always give back, and nobody should be offended by paying $20 for a project I'd pay more than that for something that way open source doesn't mean free. The issue around red hat isn't that it's just that they've always been this to just impulsively become that and expect everyone to just be okay with shifting and transitioning directions all of a sudden without any notice. For example, maybe sent to us eight snooze wouldn't be so bad if they were like yeah we're going to carry out the support agreement that we said that we're going to give you guys but after sent to us eight is done in 10 years then that's just going to be it I think that's totally fair. You're giving everyone what they promised, you're changing directions gracefully you're not doing something that's going to mess up your customers. And I think that's at the core of the issue bad communication, not keeping their customers in mind red hat is just showing us that they don't really know what open source means. And apparently if someone is upset about paying a small little license fee for something they must not understand what it means either. Yep, it's it's definitely a challenge because I I participated in the xcp ng forums, and there's people in there, someone actually complained because they wanted they wanted it removed, which was the little thing if you compile yourself because they give you all the code if you self compile everything has a little message that's part of the self compile code that says this is an unsupported version. I mean it's fair enough it's not supported it's open source it tells you that. And someone's like oh I want that removed that's you guys shouldn't have that in there that was actually requested go no this is when you get the free one it's going to do that. I mean, you can fork the code but we're not we're not going to admit a poll request to change it. You can just remove the code for yourself and recompile it but why would you it's reminding you that you don't have an open source it's just a little banner that they have on it. I don't see that as a problem it's not stopping you from doing everything because they're actually giving you the code for the full features. It's just when you don't buy a subscription plan from them, you get that little notice I don't think it's a big deal. That's really it's really not. But there's a you know you bring up poll requests and that's another risk, because there's risks everywhere. Yeah, if you submit a risk, I mean a risk, if you submit a poll request. And maybe put a lot of work into this as possible that the project may not want to go that direction because that's just not the way they're staring it and you know maybe they'll be thankful of your contribution but it may not be what they're looking for just because you provide code doesn't mean they're obligated to accept it. You know you could you could talk to them first and say hey this is what I'm thinking about doing what do you think about this so that way you could judge or gauge the likelihood of acceptance there. But let's just elite look so let's just, you know, look at all the risks here taking red hat out of the equation because I don't want to get fired up I'm going to not mention them at all. I'm not going to mention canonical or anyone so just just keeping it simple. So, if you create open source code, like I mentioned you're accepting all the risks and that includes someone profiting off of that if that's not what you want to happen. Then take that into consideration when you put the code out there don't put the code out there. And then get upset because this is happening you have to know open source enough to know what can happen to know if this is a direction that you want to go with your project or your company wants to go make that decision don't make it possibly take some time. Do you want to open source this are you okay with this and if you do want to have a profit line from this a quarter whatever. Figure that out and make sure you have research done and all that taken care of before you put it out there. It is a risk just in the same way like I mentioned you contribute to it as a risk they might not accept that code. So, here's to netgate the markup on devices and I mentioned that gate just because you because you did but it doesn't even matter if it's not get anyone who makes devices hardware devices, and they also develop the software on top of it they know the profit margin is really low. I get a little annoyed when someone mentioned to me in a system 76 review that it costs like $100 more than the equivalent windows version and I'm thinking like, but they created a custom bios for you they're supporting your computer they're supporting the fact that you're running Linux, they're putting work into this even after you buy it and you don't want them to get any money what's going on I mean you're paying a little bit of a premium for that that's just how all this goes you have a company that has their, you know what's important to you you have to make that choice for yourself for your organization, but one thing I don't agree with is put making something open source, and then you know a bunch of years later complain about the way that it or changing it but complaining about about it because you should have known going into this you should have done your research what what's going to happen or what could happen it shouldn't be a surprise you shouldn't be expecting you're going to have a Tesla or something just because you wrote code. Because there's another arm of this there's marketing, there's all the other things that a developer might not be good at that might be someone else's thing so you have to really judge all these different things and also, what is this going to undermine your design do you think someone else is going to take your code and make it that much better to where yours won't matter anymore. It's possible, but you have to keep this in mind, and there's no answer to anything that I'm saying. There's no clear answer. There, it's just a bunch of things to keep in mind for each individual person, they make their own choice I've had someone, I can remember who it was now a project that didn't become a sponsor but they mentioned that they're closed source because of reasons and I agreed with them on the call yeah that open source is not for everyone and it's not for you clearly because it's not going to work, but for other people it works just fine, but don't, you know, just rip the carpet from underneath everyone that's depending on your software, you know, decades later don't do that and don't complain because people are profiting off of it because that's kind of what people do and that's open source is open source for better work and that's what it is. Yeah, it's just matter of figuring out the right business model and it's not easy that's why we said why open source is hard in the title. It's very, very hard if it I'll tell you this, if it was a very clear if there was a clear answer here, and everyone would already have that answer there wouldn't be no podcast episode about this. We're talking about this right now the fact we're talking about this means that there's valid points on both sides that's why we have a debate if there was only one right way to go. We would just have a few people that disagree with us, we wouldn't have a divisive opinion like this and the fact that we do means that there's literally good points on both sides and we have to acknowledge that. Exactly. All right, I think we've covered this topic Jay any more comments. I have like 9997 more but I don't think we have time for it so. Yeah, we could talk about it for a long time. We didn't solve this problem today we just wanted to raise some awareness to get people thinking about it Jay's article is linked in the show notes for those you want to read some of his other topics around this. Also check out all of our other videos Jay's got a video on this as a topic and we have plenty of other stuff that will get you going with home lab and all kinds of fun open source things because well that we still we still love open source. We're not abandoning in any way we start just wanted to make sure people are aware and think about the ecosystem that they're participating in. Accept it as it is for better or worse, either love it or hate it. Yep. And that's how it is. All right, well thank you everyone and we'll see you next time.