 Welcome to the Home Lab Show, episode 110, Dev Random, lots of little updates. We have some updates and stuff we wanted to talk about and anyone try and just mush them all into one big title. So we call these episodes Dev Random, where we just talk about some of the new things out there and me and Jay are gonna dive into a little bit of the topic of the new mini little servers that are coming out, because we're both kind of impressed with like, is how fast? Even Jay was like, no, he was, this is, yeah, I was pretty shocked by that. It's faster than my Proxmox server. We'll get into that though. Yeah, so we figured that talking about some of the little server stuff, some of the updates, Home Assistant's been around for 10 years. We're gonna discuss that. I mean, this is a lot to talk about here. And I also have an interesting article about a company who saved some money moving away from the cloud. And I'll bring up how that relates to Home Lab because it's a pretty cool story. It's still in a colo, but there's still some concepts that I wanna walk through because I think the cloud has finally reached its maximum sales capacity of salespeople chilling and telling us it's the best thing ever. And then the bill has come and people go, that's a lot of bill. So we have lots to talk about here. Before we do that, let's thank a sponsor of the show that is Linode. It is a great place to host many things. If you don't wanna host it yourself, you wanna spin up a few things in a cloud that you control, you set up and you manage, they got a great way to do this. You can go over to the Linode store and just grab some of the different templates they have or start from scratch, load your favorite distribution, choose it from there, get the base image going and build what you need to build in the cloud, get it going, whether it's a VPN, whether it's one of the many projects we talk about here. Linode has been a great sponsor of the show. They're definitely fun to work with and it's where we host the Home Lab show because, well, you guys download a lot of these episodes and Linode helps with the bandwidth for that. So we thank them for being a sponsor of the show. Found an offer code down below and let's jump into it, Jay. Let's jump into it. We are gonna be taking questions as well. So feel free to throw your questions in here. We do that as part of the DevRandom here. So we like to make this interactive for those of you that do take the time to listen to us live. We'll look or watching all of the chat go by. Yep. The first thing I'll talk about though is 37 signals and if you haven't heard of them, they're an interesting group there. They're behind a few other projects as well but specifically their blog you can find and I'll actually, I can add this back to the show notes because I forgot to put this one in there. 37 signals had an idea that if they got things out of AWS they had predicted to save about $5 million a year by doing it. They were wrong about their predictions. I think they saved 7 million. So that's all, gosh, that's all. Now granted, this was built on $600,000 with the hardware they built but when you think about spending $600,000 hardware that you have to manage and maintain you're like, okay, this is interesting that that level of hardware. And you know, some people as I talk, had a discussion on a few places on this as a topic and people are saying, well, they gotta have redundancy and I'm like, even if you bought $600,000 twice because you were worried about having it in a single colo and you put it in another one, right? Your savings over time are still pretty substantial on here. And the important thing to note and this is where it relates back to the Home Lab. First, they did not add any people to their IT staff to do this. They've been doing it for a little while. They've been moving more and more things off the cloud for services they do. And they're going, hey, this is a really reasonable and it turns out we were already managing a lot of this. We were already the ones updating the servers. We're the ones that maintain a lot of our code. So we didn't have to add anyone new. Turns out managing the physical hardware which is all they really used AWS for wasn't that hard to do because hardware is generally more reliable now and it's less expensive than it was. Hard drives don't go bad as often. A lot of the stuff has become solid state so it's even more reliable overall. But it's just fascinating to hold everything that went into this and reading through and going, wow, they saved it. Now this also relates to Home Lab because, hey, even though this isn't a colo, the concept's still the same and we're seeing more and more things getting easier and easier to self-host. And I think that's just awesome. It is absolutely good news for all of the Home Lab people that are going, hey, I want to post these things myself. Is it that hard anymore? It's getting easier. Yeah, I feel like AWS is like a supermarket. You can go in there and spend a bunch of money but if you know how to manage coupons you can usually make up pretty well but you have to have like this ridiculous strategy and you have to nickel and dime everything. It ends up being very time consuming to keep the bill down. I've seen some crazy bills in AWS so that doesn't surprise me at all. I'm certified in it so I've spent a lot of time in it and I feel like it's an art in and of itself in AWS separate from other clouds because they have AWS-isms so to speak. It's just really hard to manage. So I would think in some cases managing physical hardware would have less to keep track of than having to go through every single thing. Like for example, if there's a host problem meaning like the server your virtual machine in AWS is running on has a problem they're gonna send you an email, move your VM or we're going to delete it in a week but if you don't get that email then your server disappears. So you have to make sure you get your emails. You have to make sure this, make sure that I'm not saying AWS is a platform people should avoid but it's definitely something that takes a special hand to maintain for sure. Yeah, I've had a friend and that was one of the tasks he worked for a very, very large company and his task for a couple of years was why is our AWS bill so big and can you sort it out because this company is a Fortune 500 company and I think, I forget how many million dollars a year he ended up saving them just by finding all these random things that people had spun up over time. Every department decided they needed to spend an extra server here, up a server there, up and the other problem with a lot of these cloud companies is they're always trying to get you to use their proprietary version of something instead of you know, normal and open source or standards based that way it's hard to move. So if you use the AWS specific things and Azure does the same, it's not just an AWS problem it's all the cloud companies they're trying to figure out a way to make it so you can never leave their cloud and yeah, that's not ideal, a lot of things we talk about here are very standards based and you know, it just makes life a lot easier especially because there was an outage caused because the company didn't read the docs very well they built their original product on an old style database and the new version had a lot of changes and the company didn't really understand that the database was going away in its prior version and the new way that AWS, I forget the database software name, it's one of the proprietary ones. Amazon RDS, they have RDS but then they have their own little database type in there that's probably what you're talking about. Yeah, I'm trying to figure the name of it. I can't remember the name of it either. So those. The DynamoDB or is that something else? It's been a while, but I remember the last company that reached out to me to audit their AWS I found like 20,000 a month in savings within 15 minutes it's just there's certain things you can look for and then you have your reserved instances which I believe were being sunset when I was using it and they have other ways of saving money as well. It's just a very like you can bid on other people's remaining instance time and things like that to save money. There's a lot, just a lot of weird stuff to it you have to learn. And I think, you know, you have a server make sure it works, make sure you have redundancy like we do in the home lab and it's a little simpler and to add insult to injury there's Amazon lights work or light works. Light sale. I'm thinking of light sale. I think now I'm mentioning an Adobe product I don't use. Yeah, but yeah, light sale, which is like their digital ocean slash Akamai solution which is like why can't it all just be made that simple but you know, what do I know? Yeah. And not just because they're a sponsor there's a reason we like the node it's just a simpler interface to do. It's not confusing when you set these things up and there's other cloud companies of a similar style and it's not the big ones. The big ones are always between Azure and them. I mean, they have their place but they come with the warning of some complexity. So something to take into consideration there. And Azure is more of a old school style than AWS is because AWS is all about, you know, paddle not pets and then Azure, obviously the same strategy would work well there too but you generally it's treated like an online version of their virtual PC software which I believe they renamed but it's essentially your classic virtual machine style in a browser was AWS is just completely different. So again, if someone wants to get into AWS you could make a living just by saving people money and finding things. So there's a lot of money to be made there. So. Now, absolutely, HomeLab and this is one of those things that I haven't done a video on yet but I plan to soon. HomeLab is now 10 years old as of September 17th of 2023 and that's awesome. 10 years old and it is now the second most active as of today at least the second most active open source project on GitHub. That's amazing. We're talking about Home Assistant not HomeLab, right? I mean, HomeLab, yes we're talking about. Yeah, you said HomeLab is 10 years old like is that when the term is first born? Home Assistant, our favorite thing in HomeLab among the favorite things of all HomeLabbers, I think. Oh yeah, it's definitely one of those projects that is somewhat of a unicorn because of, you know, I find it very well designed. It's got a unique interface but it doesn't release for me it didn't take that much time to learn it and it's just consistent between config files and everything. It's just a unicorn because I feel like it's very well designed that it makes home automation more approachable for the average person that just doesn't wanna code all of this themselves or rely on, you know, 10 different providers for 10 different things that wraps those all together in one interface which is just brilliant. And so Navakatsu, the company behind it is five years old. So they started this open source project Navakatsu became one of the, you know big contributors to this. There's no investors and development of Home Assistant is stable. Those are things that I think are really important because the no investor is how you stop the cycle we watch companies go through. And open source companies are generally provided they don't get large corporate investors. IBM Red Hat, true. You don't watch them go through a cycle of destroying users. And if you wanna look it up Corey Dockdraff has really been pushing this and I love his term, the enchantification cycle of a company. And there's ways to avoid this. It's a documented process companies go through as they take investors they treat their users nice to get those users they lock those users in then they abuse those users as much as possible to extract every ounce of value while providing less and less services to you until you finally watch the company die and you have to switch the next platform which will just do the same thing to you again. There's one of the big things behind open source is this cycle gets broken because of the code out there. And generally these don't get a lot of like the open source projects don't get big investments and things like that. They usually eat like this homegrown community base they require sometimes a company be behind them that's a steward of it but as long as those companies don't take massive investment they generally stay the course really well. SureNAS is an example of it as well IX systems competes very much in the enterprise storage space that is dominated by venture capital and large companies like Dell and Hewlett Packard and the mega-courts of the world and people who use these products especially because I recently had to interact with a several hundred thousand dollar three-par system that the customer is not happy with at all. But it's like the whole customer experience but it's terrible and that's why they contacted us as to get over to TrueNAS which is less terrible but back over to Home Assistant I just really think it's awesome. One of the things you're really focusing on right now this is right from them in their blog post which I'll throw in the show notes here the open home vision is updated and now is about privacy, choice and sustainability and I think that's really cool because we already know the way many of these other devices we wanna work through other assistance especially when they're proprietary the way they get in there and way they monetize is figuring out how to monetize the fact that you exist and what information can they gleam and sell to some other place? How many light switches in this person's house? How often do they turn their light switches on and off? I bet we could sell that data. And like that's like their business model because they have to repay investors so their pressure from investors is what did we learn about how often Jay turns something on or off? Well they want you to let them tree let's just link something to dev random literally dev random on this out of system to give them some random data and just keep sending them that until they have their fill of random you know gobbledygook that they can do whatever they want with be the ultimate home lab oppositional mindset. Yeah. Kind of funny to see if that was a possibility I'm sure somebody has done something like that before and then even Windows users it's like there's been articles lately that you know Windows users go through a lot too because of the things that come along with Windows that's worse than it's ever been so it's just yeah I kind of agree and should have occasion. Yeah that's just it's a mess but hey that's the whole idea though is that they're doing it a lot better over at the Home Assistant it's like it says a big project I highly recommend definitely a big fan of what they're doing over there Jay's dove deeply into it customization it's a bit overwhelming when you first start the good news is there's a lot not me and Jay specifically but there's a lot of YouTube channels dedicated just to this and it's wild because I don't think there's a more complete ecosystem for everything you could possibly tie to your Home Lab if you want to modify your Wi-Fi you want to modify your firewall rules based on you walking into a room and changes the firewall rule the Home Assistant is the only thing I can think of that would actually allow you to do that you can actually have it change firewall rules and PF sense modify your Wi-Fi and Unify then combine it with camera systems and everything else and then build trigger events and you can say modify my firewall adjust the temperature in this room because you can tie it to a lot of different temperature sensors it's one of those things that's why it can be overwhelming because I don't know what the limit is or if there is a limit for what you can plug into Home Assistant it's always blowing my mind all the different things that are out there so yeah and if you feel overwhelmed I feel like at some point you kind of like unexpectedly get to a point where you kind of notice this pattern in the UI and then you notice like this control acts the same as that one and this one has the same configuration menu even though it's a different thing there's some consistency there and that kind of helps because at first it's overwhelming but then it kind of becomes on overwhelming fairly quickly when you get to that point but if nothing else you can stick to one thing and then complete that one thing before doing anything else don't try to like multitask and have I'm gonna set up the light bulbs and I'm gonna do the smart plugs and this and that next thing you know you have a bunch of things that aren't done you know just focus on one thing get the light bulbs done or the switches done whatever you want to start with and don't move on to anything else so you finish that one thing and then move on to the second thing and then just be a single task kind of mindset and eventually I think that kind of will make you more productive but sometimes it's easy like oh I can't figure this thing out so I'm gonna just take a look at this thing and then next thing you know you have like a bunch of unfinished implementation so I think that's probably key another tip I have is to set up your rooms and everything get that done first so you have like the layout of your house and the room setup because you can add rooms to it and that kind of gives you a target just think about you know you have a den you have a finished basement whatever you have how many bedrooms put those in there and you have that foundation and that's easy you don't have to configure anything you're just telling it what rooms you have and then from there you just populate the rooms with that one thing that you're focusing on until that one thing is done and then you move on to the next thing and then eventually you're just trying to run out of things to do essentially which you won't because you'll find new things to do but that's part of the fun but at the very beginning at least try to get the foundational things done Yeah and it's the way you can set the multiple dashboards is cool because it's friendly for your non-tech people you may cohabitate with because you can build dashboards tie it to their phone and then let them control it and you know my wife's less technical also she does not need access to everything in my studio and turning it on and off I believe she would I believe she would use it and she would be very used by it I think it'd be pretty funny Yeah why don't we turn the lights off and then yeah Lights around lights around it yeah it's kind of fun Next topic is the we brought it up at the last home that shows I think some people asked about it but it's the 45 Drives HL15 you're gonna see a lot more about it from several people from the Creator Summit and I'm getting one as well I believe I don't know we haven't confirmed yet whether or not Jay is but hopefully but the new HL15 you can 45 Drives is really committed to engaging and interacting with the homelab community as well so they've built a name for themselves in the enterprise storage space Linus obviously made them somewhat popular because of the massive amount of storage Linus has bought with them and talked about them a few times I've done the same I've talked about building petabyte servers with them and we commercially used them a lot but what's interesting from them as a company is they put it all on GitHub they are huge on making this more like they've been really working hard to go how do we take one of our premium products that is gonna be a little expensive for the homelab people no doubt I mean hey great if you can find one of them used on eBay or something like that but we're looking for something consistent and higher end yep but the the fact that now you can well pre-order is as close as they have right now I don't know when the ship date is they're building this one called the HL for homelab 15 is a 15 bay premium server and like me and craft computing we're talking about this and it's well those things can you find something that's this high quality there's lots of companies that stamping out cheaper things and whatnot but we were talking about like how do we get something affordable that we can buy consistently not trying to hunt down some used thing on eBay or something like that that's a nice quality product that is very customizable built on open standards this is one of the things that's really cool about the 45 drive system is a this is not a weird proprietary backplane this is a 15 drive direct connect backplane they'll usually throw in an LSI controller card so something common that fits in a PCI slot and pop whatever motherboard you want unless you're crazy enough to be like Jeff Geerling and try to figure out how you can shoot in a raspberry pi into a 60 drive thing watch his videos on that to see how that went it's a that's an interesting experiment but the future maybe not to not that today but there is maybe a future for arm controlled raid systems too long didn't watch Jeff's video one it's a fun deep dive but also it turns out not to be the most practical thing right now to do but I'm excited about this I think a a premium quality case that you're going to be able to customize and upgrade with you is for your home setup is nice I think it's a big market that's that's underserved I mean there's some people that would love to buy a server just for learning and not pay a huge amount of money for it or build it up to what they want and sometimes that could be a barrier of entry someone wants to get into homelab but then the servers that are available on ebay you know the off lease servers that you get for you know usually a cheap amount of money might not might be too power hungry that could be you know too old for what that person wants to do getting something newer and you know sometimes the processors are on newer things will be a little surprise you as we'll probably talk about but I think most companies should have something for homelab a chassis maybe a server that you can buy that different you know because you only get like enterprise servers and they assume that everybody buying a servers for enterprise I don't expect Dell to serve this market or anything but I automatically think of tuxedo and system 76 and a number of others that could easily serve this market because they already have a lot of that audience and the engineering people anyway so may as well give them something and I guess 45 drives is beating them to it though and I think we're especially the 15 drive part comes in drives are generally speaking getting cheaper and let's let's just cut to the chase people are building their home media libraries because Netflix temporarily certainly slowed down the fact that people and change people's minds like oh man it's just it is very basic inexpensive few dollars a month I have access to all this content and then came the lawyers and now you have to subscribe to like a dozen streaming services that are more money and you don't even that it's like it drives you nuts like okay I have access to my content or I don't Oh well the first three seasons were produced by this person who owns the copyright to the first three seasons the next three seasons are copyright here so you got to buy a subscription to watch them here this this just drives us crazy and it's the concept of I want to buy my media and own it I pay for it I'm not actually endorsing piracy here I'm talking about the fact that many people in reading our friends over at have one of my friends what's his name I can't write two guys the two guys tech he did a video and I didn't watch it yet but I've seen how popular it was right away about how to rip your blue rays and it's like you own the media you've purchased it you would like it in a digital format you need somewhere to put all the digital format because you know I got to admit it feels a little archaic putting a disc in a machine my my kids aren't really for that I'm not even for that so this a 15 drive server seems like a great place to put that on there and having enough horsepower to you know play that back at 4k I think that's that's awesome Wendell was even talking about how you know he has people he talked about this in a live stream which actually made me laugh because I haven't thought about this in a long time older relatives of his that would still like to watch Green Acres and I was like oh man I remember that my you know my grandma loved that and he's like yep and if you owned if you bought the DVD collector set of Green Acres you could put it on Plex and make it relative Plex for your whatever you choose there's some controversy recently with Plex yeah whatever your media playback to yeah whatever media playback service you'd like to use it is locally hostable and and can broadcast to your TV now you're able to do that so I think this is going to be a really cool box to put it on so I'm excited that they're really diving into it and by the way they're not just a company it talks about open source when it comes to hardware they their Github is filled with all kinds of install scripts setup scripts all kinds of things that they use that you could use as the home lab so do check out 45 Drives Github on that too yep definitely someone says running 24-7 with skyrocket the electricity bill unless there was a company that would run out the server space that's going to be that's where things get a little bit tricky because it depends where you're at electricity prices very greatly obviously sorry in the European market I know what it's European market is obviously substantially more expensive but if you plan to have your media at home I don't know how to have your media at home without something like this like if you although then going back to technically if you're worried about power you wouldn't rip your blue rays and keep them all running system well another thing to think about too in my opinion there's really no need to keep your home lab running 24-7 unless you have it's backup night and that's the the time that off-site things go but you just get a raspberry pie or some other computer to wake on land around the time that your alarm goes off in the morning when you wake up and then that powers on everything and then you know when you go to sleep just have a schedule tasks shuts it down and then you could save power over the evening but if you have power efficient equipment I mean I don't really know how power efficient that is yet because I have yet to test one but newer technology is often more power efficient as long as that's the case then it might not be as bad as you think especially if it idle is really low but at the same time sometimes I wonder why you know people in home lab keep their servers running 24-7 when they don't it's not like they have 21-7 operations where they're going to watch green acres at four in the morning I mean probably not in there's some you know controversy where someone will say it's unhealthy to power your servers on because it uses more electricity I have never seen that as a problem during my entire career so far unless maybe you just have really bad or bad drives a bad batch or something but you know if you're not if you're not awake just shut it down and have it have wake on land power it on for you actually they just talked about this on one of the last episodes of two and a half habits and there is a stress it's not the power problem it's a stress that the drive motors are under but I don't know I don't think they were 100% clear if there's ever been a true long-term study of do we turn this hard drive on an awful lot or we just leave it running one thing that has been interesting that's come up is the fact that SSDs especially the higher the higher capacity ones are using more electricity than the spinning rust drives so that's something else to think about so the drives themselves add an idle are actually very low power so if you come back to what motherboard you're using and let's dive into that as a topic that's where a lot of your wattage goes especially if you grab an old server these old Dell R720s and things like that I've talked about them in the past and I think we're finally going to see that the huge server markets could be coming a less popular option because of the power and because these little things and I happen to have one of these and I'm testing is called an Ace Magic AM 20 has a Ryzen 7 730 7.735 H processor in there and these things are really low power wattage the new series of Ryzens and some of the low powered CPUs they have blow you away with just how little they idle at and that's where a big change the fact that a lot of this hardware now idles at such a low wattage it becomes not as big of a deal and as and me and jave talked a little bit about this because like Mac it I hate the fact that I like it so much but I got a Mac M1 laptop and because they use the newer modern arm architectures and I know this goes off on the rise and we'll come back to that we have as a community a tech community I should say we have found ways in technology we can incredibly extend the range of batteries by really limited power usage and then you take that same technology and plug it into a server and they go wow we can actually have these things spin up as needed to a high wattage powerful render whatever you want to render but when they're not doing anything we can put a bunch of course to sleep and I think the new engineering around a lot of that and a new efficiencies are really going to start shaping things up in the server world but also greatly devalue these old power hungry beasts that were there because when you're looking at and I'm holding a handheld on there some of these handheld gaming things me and jave both have recently acquired some of them like we can play some of these games on these things for hours and you're like it's amazing how little power some of these new chips can use on there this is what's actually making things like the steam deck and some of the other things very possible it's just this whole circle of we're figuring out how to be more power efficient but still deliver and I think that's where it gets really really imprendous oh yeah definitely the device I'm testing is Ace Magic AM20 though it's one of those generic things if you will it's one of those I don't know what to call I'm not backed by a large company I hate to just say simply made in China because that most of the stuff is all made in China it's just about who actually manufactures or supervises the manufacturer I think she could say so you can say like the Mac is made in China a lot of Apple products are made in China but they're supervised very closely by Apple versus these companies may have some lesser quality because they're less supervised or engineered by a larger company but one of the things that we've talked about in the home lab and Wendell's brought this up a few times I think this is is a cool idea you can buy several of these and cluster them together and it works rather well and now you've solved your redundancy problem in case one of these has some bugs or flaws or just fails in general because I'm actually on my desk here is one of those cool Tom boxes that failed in an unusual way which is going to do a video about the unusual failure mode it has because it's still partially works so it's still partially useful but this is a problem we've seen a few times where the network interface card just dies but it has four of them soldered to the board one out of four has died and we have we don't know why it's died we know it's happened to a few of them we just can't put our finger on the why because you think all the network ports would go bad I mean they're individually ports so they each have their own chip but why do one of them go bad it's not the same one each time that goes bad so it's a puzzle maybe the solder isn't a good job or something and a certain body heat just wiggles it or gets it you know loose enough I don't know yeah it was on a UPS and everything else and but I think it just goes to the the boxes are cheap and putting another one in was also an inexpensive solution but you know it's something we don't see when we have the higher quality boxes it's not a problem we really deal with what's confusing about it is the link light is on but when the tech went out to the place he unplugged it and the link light's still on so he rebooted it and the link light's on matter of fact as soon as you plug it in the link light just turns on on that port no matter what so that's the failure mode we've seen we've seen this more than once because as soon as he said I unplugged it with the link light still on I'm like oh yeah that port's bad just reassigned to another port and problem solved so it works with the other we they had an extra port they weren't using so that solved it in the short term well that's good at least out of abundance of caution we replaced it anyways may as well yeah it's amazing how powerful those things are because because the one you're using the CPU outscores the CPU in my current Proxmox servers the downside is 2.5 gigabit ethernet which is good normally but I have 10 gig and it has less RAM but the CPU though oh man I just think my VMs have actually run faster on it if it can hold all of them because of the lower RAM but yeah I mean that'd be really fun to test out yeah the one thing that's slowly getting here is in this is the problem with the lace magic box it'll complain about is exactly what Jay said the two and a half gig I don't know why it's taking them so long to get 10 gig I don't know if there's just not a surplus of it available for them to build into the boards and I'd be fine if it was SFP because SFP 10 gig has been around forever and and you can buy relatively inexpensive SFP PCI cards but this is the challenge none of these seem to put an SFP cage on any of these boxes that offers 10 gig and I'm like ah and this drives up the some of the cost you may have in a home lab because you're like oh I'll just buy a two and a half gig switch then you price it out and you're like oh that's a lot more than I expected for a two and a half gig because the two and a half gig you're coming down a lot matter of fact it's an unmanaged switch but still I'm going to be reviewing it as part of some low cost home lab parts I have this little switch that has four two and a half gig ports it's made by Euling I posted it on my YouTube channel if you look through the community tab it's interesting because it's got two 10 gig ports two 10 gig up links and then four or five two and a half gig up links on there so it is $69 so you get two 10 gig ports and a few two and a half gig ports for 69 bucks I'm like hey that's out of bad price and I think that kind of brings things back to being affordable to have good connectivity in your home lab absolutely absolutely it's just amazing that I think things are getting faster again it seems like I mean everything's always getting faster but it seems like not as much faster but now it's just these small PCs have a really good processor in those maybe not the best network ports apparently but you know at least they have to compute yeah so that's definitely it is getting there that's one of those things I think it's going to reshape our home lab a lot is all these more powerful devices and the Raspberry Pi was cool because that was like the first arm shift that really pushed it out there there's a lot of compatibility problems when you get outside of the Raspberry Pi but now that we're seeing more of a similar you know like these low powered risings I think it's true I know I'm talking arm versus x86 but I see this kind of lots of low powered things that is very accessible to a lot of people in the home lab that if they want to build something of substance you know build a Kubernetes cluster build something that can really be a learning platform or just your own platform to tinker with and build your next idea you don't have to or reach out to some high-end expensive or even going on eBay which has a big power cost of these use servers you can get more and more you know just affordable things from this we also have that company sending us the Ace Magic Company one of their they're going to be like sub $200 ones because I want to play with it for routers we also have Zima boards I've reviewed those and the new Zima Blades a few other of our friends have some of those that you'll find a lot of YouTube videos popping up pretty soon on yep it's coming yep all kinds of changes all kinds of fun changes is there anything else we have for the dev random today I don't have anything on my end I'm not seeing yeah questions so far one thing I'll bring up and I I just don't have any easy answer for this at all PPOE we just don't see it much here in the United States so people ask me constantly it's a super popular question because I guess there's not a lot of documentation on it but the reason there's not a lot of documentation I just don't run into it in the business world at all even in the home user world we used to use PPOE over DSL like 15 years ago and it kind of died out it may it may exist some places I'm not saying it's it's not in zero use but boy it is one of those challenges where how many people people ask me a lot because they always say well there's a bug in the PPOE with PF Sense or with this router that router and I'm like yeah I don't know but I think this is the same problem some of the developers have if they don't have enough exposure and experience with it they also can't write a better PPOE system because it's a niche problem I don't understand why it still persists because it seems to be Europe and Canada that if if I had to guess because there's someone in the comments here tell me tell me if you're from Europe or Canada right okay then some they say in the EU in the EU so Europe you can't have DHCP huh yeah I I don't know why because DHCP it turns out not just an American thing so it it is a standard as an RFC it can absolutely it can cross borders I thought people were we're setting up PPOE before like PF sense or something it's been so long I'm trying to remember what the go to used to be but feel like I'm going to butcher it because it's been so long it's probably been 10 or 15 years since I last had to do it and just like you said I know it still exists but I wonder if AT&T DSL still has it but and I remember that being the last time I've seen it yeah that's pretty much where I always seen it was AT&T DSL yeah that was a very long time ago but I imagine it does exist so I see someone saying in using it via fiber which is weird because all my fiber clients zero of them have PPOE so that's interesting someone says the ISP at work was to use PPE even under fiber network and that is in Alaska never really had any issues with it on PF sense interesting yeah that is pretty interesting feel like more research is needed after looking to how we use it just still is as well as things because I've had requests like I set it up and tested I'm like could I build a PPOE server yes but if you're trying to troubleshoot something if I followed standards and built it I'm not likely to have the same problems people would encounter because there's people who have encountered problems with the way their ISP implemented something but I mean Jay have you ever heard of an ISP who even does follow standards properly you don't think kind of color a little outside the lines generally when they set these things up they do so many things and sometimes I squirm you know I feel like everybody should be concerned about privacy but they'll be concerned about privacy while ignoring their table modem because I feel like their internet provider is going to be an egregious privacy invader and you know we need to focus on all the things but but you know they always get away with things until somebody figures out or looks in the packets and sees what they're up to it's just a constant it's a meme at this point yeah actually someone says that they're a European they work for a European ISP we don't deploy new PPOE networks anymore everything is now DHCP with DHCP tag we're good they're moving forward yep that I wonder if some of the PPOE that's being used it's just an ISP that use it as an authentication means to prove that you're a subscriber and they just don't have any real you know need to move off of it or something because it's validating the subscriber in some way but like you said there's better ways to do that now yeah yeah it's one of those you know a lot of you have a choice unless you manage the ISP if that's how they have it set up and you have to go along with it and you're pretty much deal with the interconnection you're given you don't get a choice right well thank you for joining us on The Home Lab Show much appreciated and leave us same comments leave us some likes that is always great head over to thehomelab.show you can contact us there we love hearing from you we like to do the listener feedback ones and that is all take care everyone thanks