 Welcome to the Home Lab Show episode 120. How are you doing, Jay? I'm doing great. How are you? Wonderful. I figured it's been out for a little while and it's gotten better. We're not just riding hype trains here. We are talking about ways you can actually use this in your Home Lab, and we'll talk about that a little bit in this episode. First thing we do want to announce is, one, we still need a sponsor. We got to figure that out on our back end. Me and Jay are really good at creating videos, and we're trying to get better at some of the business side of YouTube or creating content. Yeah. It's funny how we could just be so deep in the content. We're hyper focused on creating and delivering. Yeah. We're focused on that, but then when it comes to, I don't know, we just haven't even tried to get a sponsor. So it's not like we're sponsored or tried in our part. We haven't even emailed anybody or anything because it's like we have so much fun with the content that it's hard to pull us away from it. We're hoping that by saying something, I'm going to go, Hey, I like these guys and I'd love to sponsor them, and then we're going to have to do any more work, but I have a feeling we're going to have to do some work. So what you, our little ramble here was the insert ad read. That was where you would have heard it. Yeah. We appreciate everything that Akamai has done. There's no hard feelings. We still use them because we always, and this is a good example of this because as of today, Akamai is not a sponsor of even my channel. However, I still use them because when I say that I actually use the things that sponsor the channel, I mean it. I'm literally still using them. So their channel, their YouTube channel, I think hit over a million view or subscribers at this point. So I think they're just flourishing right now, but we appreciate everything. And yes, we do use the things that sponsor us. And the fact we still use it, Akamai is an example of that. We're still hosting it. And they make it, that's why we like them as a sponsor. They just made some changes internally. And I realized I posted this and it said PM. Let me make sure there's not a second. It should be this. Yeah. We're live now. There's not another one coming up at 11 PM just for people and they were wondering or seeing the notice go out. It's yeah. So this is the live show. There's not a second one at 11 PM. Me and Jay are less coherent at 11 PM to accomplish tasks. But it's like, why are they so discombobulated? We really do enjoy making content so much that we love our job and it's hard to do. It's a 12 hour live stream. We're going all the way till, yeah, all right. We should probably do that one day anyway. Yeah, we will do a late night one maybe one time. I've done some of my vlog Thursday ones late. But speaking of one more topic of the node and we'll cover a little radar for we jump into the AI topic. I am upgrading my forums. They've been on digital ocean for a while because I don't want to move them. There's a lot of steps involved. And then I didn't want to disrupt anything but to the point and for those of you listening, I don't only have an easy way because I don't collect people's email addresses or anything. I don't have a way to mass announce a downtime easily. I mean, I could. I think there's a way I could probably mass email people. I don't know that that seems like a good idea to send out. There's like 6,000 people in there sending out 6,000 emails to let people know the forums will be down by a half hour roughly while I switched them. This is gonna take about a half hour to flip it. It doesn't seem like the best 6,000. Well, the only thing, I mean, this is kind of like a side topic that I don't, we don't necessarily need to go, you know, another episode topic, but I'll just mention some of this real quick because when I say DNS is a challenge, everybody knows that. Okay. But specifically it's some things are harder to be graceful because of DNS. So if you have to wait for DNS to flip before you finalize a migration of one thing to another that prevents it from being seamless. Now, there's other things we can do like have a temporary, you know, $5 a month instance that literally just has Apache on it with one static HTML pages says under maintenance and flip DNS to that first and to let everybody know, but then you still have to flip DNS to the final version and disrupt everybody because at some point that switch has to be made. And then when you're dealing with let's encrypt, you have to be able to authenticate the site, obviously, but to do that, if it's a DNS challenge, you have to have that done first, which means people are gonna get downtime. Now, obviously there's ways to architect this so nobody has downtime. And yes, we do know that, but the amount of work that takes versus how little work by comparison the migration itself takes, it's like, it doesn't, it's hard, you know, to justify that extra time making content. So if you guys are listening live and there's downtime in the forums, it's like I said, it's gonna be doing it the DNS way because, you know, building a seamless failover system to avoid a potential of up to 30 minutes of downtime. And the problem with putting a display page like someone had suggested is the DNS is the problem. Some people, while I'm waiting for the let's encrypt certificate to point at the right place are gonna get an invalid cert. And yeah, so you'll have an invalid cert error to see that the site is in maintenance mode with the announcement. And depending on how it propagates, so because it's gonna hit certain servers first, I don't know when it's gonna hit you individual user when it hits your DNS server. What if you were in the forums and you have a cash copy of it? The TTL says it's only gonna last some amount of time, but maybe you've modified it or your caching's modified it, it's gonna take you longer. So these are the nuances of the backend of things. Yeah, DNS is complicated. Like somebody mentioned in the chat room, you could just cut the time to live down. And that helps, but it's still, even if you drop it down to five minutes, there's still people that are gonna hit it within five minutes. So it's, I mean, it can be 100% of some cloud platforms because they have some services plus, as I call them, where they get a service like DNS or certificates that you could get anywhere, but they have additional features on it that can help with that too. But it's one of those things where it's like over architecting versus just get it done and people will forget it ever happened anyway. One more little piece of a rant I hear that I think is worth mentioning because me and Jay talked about this last night and Jay still has a great series on Proxmox and I have been updating my series on XCPNG. So for those of you, and I know it's really popular in a home lab, but if you didn't know just the other day, one or two days yesterday, camera was yesterday day before, it's all blur to me, VMware has officially, not like these back channels like rumored, but officially killed all their free version of their software. They've eliminated the ISO downloads and we know this is going to be big for the home lab people going, now what? Once they started killing off the licenses, the free licenses, the perpetual licenses and the home lab licenses, now we're all gone as well. The downloads are all removed. People are gonna go, what do I do now? And Jay's team Proxmox, I'm team XCPNG, but I think both of these are solid home lab solutions. So whichever one makes you happy? Because my frequent, you know, response to people when they start going, well, why don't you use this one? And I'm like, I've got a video explaining why, but if you don't like my reasons, that's fine. Use what makes you happy. I'm not here to tell you there's only one choice. I'm telling you it's a beautiful market when there's multiple choices for things. This is a really hard choice between XCPNG and Proxmox and I still maintain this to this day. I like both just because I have a bunch of videos on Proxmox doesn't mean I'm team Proxmox and Tom and his team XCPNG, it's not like that. I honestly feel like if you want a solution you can flip a coin between the two, whichever one you land on will serve you well. They're pros and the cons make them in my opinion completely even. Yeah, and someone brought up because there is a free offering by Nutanix and that's a free offering, but I think there's no sane choice to choose. It doesn't seem like a sane choice because Nutanix is also a large closed source publicly traded company and matter of fact, they're so small that if they became a nuisance to the VMware path of destruction that Broadcom has put them on, they could just buy them. Matter of fact, I realize looking it up Nutanix even though they're publicly traded and they are a big company, they are a pocket change to the size and scale of Broadcom. So Broadcom wanted to swallow them, they could. So that's on Nutanix because that has been popping up a lot because like now that the rubber has set the road so to speak and we know that, well, they've eliminated the free versions of VMware, people seem to look for the other one. And I think I'm preaching to the choir here. I know the Homeland Audience is huge and self-hosted, huge and open source and it gives you a better decentralized working model where you're not dependent on these companies and the whims of their venture capital overlords. I was just saying that was. I feel really sad about all the people that before this happened, like many people do in HomeLab they set something up at home because they're using it at work. Maybe they wanna get into virtualization at their entry level or maybe they wanna do a career switch or they just wanna prepare for a certification or something like that. And I just kind of feel like there's gonna be a lot of people that have this on their HomeLab for no other reason than they just wanted to learn it. And now they have a work and HomeLab decision to make which is even worse. Yeah, and me and Jay are huge advocates and we're even gonna be doing another show on Jay's channel about this, about owning your own media. That's something that is upcoming as well. We also have some special guests coming on the HomeLab show then or all the same thing. I believe this is how we make a better internet. This is the old internet, bring brought back to life the older version of it if you will where we always had decentralized and self-hosted and not interdependencies on a lot of these companies. But yes, that is, as I know I'm pretty sure to acquire on that, but that's also where Chatch FDP, it brought the whole AI models popular, but do you really wanna give your data to a private company? And the answer to that to me is no. And I'm pretty sure Jay agrees with me on that. You wouldn't just upload all your data there. Although me and Jay certainly use some of the publicly available private companies that do things like help us out with certain aspects of how we do our content creation. I've certainly used the Taja AI system. I've talked about that before. It's a tool for creators that allow us to create all the summaries of our videos and help document and put little timestamps and chapters and that's cool and it's public information anyway so I don't mind. But now let's talk about how can you self-host these? And this is what's kinda cool is I've been playing around with them a lot more. Now it takes a lot to train a model. There's a lot of different costs I've seen thrown out there when you wanna just pump these models full of data but the good news is there's a lot more companies that realize, hey, if we open source these models we get community participation. So even though the models may have been trained on these and we'll get to how that works in a moment but more importantly, they're completely once trained usable to you to self-host in your own private system. And we're gonna start with one that I found extremely easy to set up and that's the O-Lama system. It's essentially a framework to run a bunch of other ones and I like that it's got a llama in it. Like it says. When I've had a llama, remember that? When now? What's that? When now? Remember when now? Oh yeah. So yeah, you're right. I just, I was gonna say I aged myself it doesn't feel like that long ago but I'm starting to think it kinda was a while ago wasn't it? Yeah. I mean, when have like still be a thing for all I know but we used to love that software back in the day and then they had a llama slogan or whatever so it's always gonna be a reminder. Yes. So currently and I'm over on the O-Lama site right now and by the way, I jumped a lot of links including a lot of good YouTube videos that break this down including one of them that shows you how you can run this on a Raspberry Pi 5 by the way, AJ has a new video on a Raspberry Pi 5 and I think that's what is really cool. I see someone else talking about local AI I haven't tested local AI but one of the cool things about the models that are out there is you have different models for different use cases. And one of the popular ones now has become especially for those you in the homelab it might be the most interesting you isn't just a general one but a more specifically tuned one. Now the advantages of a tuned one it's gonna be obviously tuned and focused on what you wanna do so you've got the code llama and the cool thing about like a code llama one they're not gonna take as much memory doesn't take as much horsepower by the way you don't have to buy a supercomputer to run these that's why I brought up that first one about being able to run it on a Raspberry Pi this allows you to start figuring out what do you want, what's your use case is it you just wanna have a conversation with it or do you wanna dive a little bit deeper or you want it to be kind of your code assistant but you don't necessarily want your code to be given to some of these third party companies not to mention the subscription you may need for it so being able to run these locally is great. Now we'll go a step further to something I think people really like is the fact that not only can you run them locally they also have versions of them that are uncensored and what that means is if you ask them to open the pod bay doors they can't deny you they can't tell you that there's a rule that says this this is something that's become really fascinating to me is that they've essentially taken and modified the biases in them or removed some of their safety restrictions on them I think this is really a good step forward because it's not necessarily that I'm encouraging or suggesting that people are using it for nefarious things but sometimes the safety parameters put around them may stop you from doing what you want to do especially if you're working in malware research you're not trying to use it to hack something you're trying to better understand security and maybe if you ask the question like what are different attack methodologies or what are some of the processes or let me like some power show code for this particular type of attack you're doing it as a learning experience so you can understand the code, understand the vectors and of course you're not gonna be able to do this with the public facing ones and I think that's a good place to start is with the Olama if you're looking for like a really like one liner copy paste and by the way you can run this on your local desktop and then accelerate it it even auto detects if you have an Nvidia card so you can accelerate it because these will work without an Nvidia card without a GPU basis on this so getting started with this is actually that easy it's kind of a copy paste one liner with the Olama model and anytime you call up another model you can then from there it'll just go wait, is this model installed and if it is great it'll start answering questions but don't it'll let you know how long it's got to download before that model's installed by the way as long as you got enough hard drive space you can keep installing all the models and keep updating them I think that's pretty cool too you can grab them it will also switch between them so you can start instances of them and they started modifying the API and I don't know if this is fully released yet but I've seen it in the announcements so the API is compatible with I believe ChatGPT now they use the same API calls so if you see a project that works already with ChatGPT it's easy just to point it locally I have not tested this but people co-pilot and things like that is out now for some of your writing you can now attach it to your local models and I think that's just a really neat instance Yeah, absolutely that sounds pretty cool I've never actually heard about this until today Yeah, I told you it's a rabbit hole I got sucked into and I was so surprised with how well it worked that's the part that really shocked me Now to go a step further I've left the links to like I said all these things I'm talking about I've left links too because I'm not an expert on it I'm mostly and me and Jay wanted to intro all of you to this to kind of get started it's a great place to start poking around that's why there's so many GitHub links and they all have nice installs one of the other things that comes up a lot is how do I talk to my or look at my private data my documents once again there is a private document one where you can then point it at your documents and interact with them because maybe you've done a bunch of different research and you want to be able to pull or attract data from that research there's a private tool for that and the one that fascinated me and there's a this girl did an amazing video on it I watched it I have not tested it but I will I'm also thinking this is where a lot of people are going it's a tool called a crew.ai and the concept is really simple so to give you a little bit of an overview of the wonderful AI I don't think AI is the I don't get over hyped on it I don't think it's a savior for all things I don't think it's coming for everybody's shop but I do think it is a great assistive tool for example I use it for things like regex I've talked about this and maybe I'll do a video on how to use an AI tool to write gray log grok patterns based on logs you know I give it logs and I say hey let's go ahead and pull the grok patterns out of this and write a copy paste so I can import it right into gray log this is a great use case so I think I find it to be a really good assistive tool well that's where this whole crew.ai system comes in where each one of these gpt's you you have to prompt them essentially and what prompting means is we're going to set the biases we're going to tell them hey you're going to give me brief responses you are an expert in coding or you're an expert in bash it actually helps to tell it's like you got to give an AI a confidence booster you're going to be an expert you're going to give me short descriptions of things you're going to answer with code I don't need explainers on the code or you can say give me long explainers with each one of these code functions mean by the way when you write the code put all the code comments in there so I can understand it later however you want to prompt them now what's really cool about this and where their crew AI that I mentioned here comes into play and it's the idea that you have to prompt each one which makes them good at a task then you have to start the prompt again crew AI is a really cool tool and it can be pointed at these local ones where you can string that together you can create a prompt so you prompt the first one with a certain input like you're an expert in these type of decision making here's the data I want you to process then you connect the output of the first GPT with its prompt to a reprompted new version or maybe in a different GTT it has a different level of expertise a different prompt you take the output of one you chain it to the next and you do this again and again until you get a better decision it's kind of like putting a team of people together that have different levels of expertise to try to come to a better decision I just think that's a really fascinating use case you're like wait we're going to pipe it through here it's kind of like we always joke and there's a million of them and beams about a full stack developer and they're always good at back end they're good at front end you either get a really pretty UI with a broken back end or you get a really good back end with a looks like it was built like an engineer UI that you could barely call a UI what if you could train one side of it to be really good at writing code and one side being really good at building pretty UIs and because the biases are different chain them together to get a better code output to develop your application that's kind of the concept I'm working with it it's really cool it is I like putting config files in there just obviously make sure it's stripped of anything that may have PIA I don't know why anyone have PIA in their config files but basically just find flaws like put your SSH host config in there and like find an issue with this that someone might use to get into the server and depending I mean you might get something like very entry level depending because I know there's chat GBT3 there's four there's other ones like we're mentioning today as well but sometimes just think about having another set of eyes and we can't we can't have the mindset that well AI didn't find an issue so it's flawless no it just means it didn't find an issue but if it does find one then we know about it we could fix it yeah and that's what's kind of neat you can also do things and take a lot of the outputs you have and say explain this to me and just drop those information in there and what's nice is when you're doing this in a forum you're always redacting you're always like oh let me redact any private information because this was inside my home lab maybe there's some private data maybe there's some certificates in there and I'm worried about it one of the advantages you get with cellpost that worry goes away I can just say I don't take the output of these error messages pipe this log over to chat GB for my instance of Olama GPT what is the output of this code I'm not understanding what this means you're not worried about revealing any secrets or any private keys as Jason you may have in there I think this is a really solid use case for the home lab and it gets you started with less risk involved in terms of the risk of oh no I've accidentally uploaded an entire financial document to to chat GPT and now everyone has a copy of it yeah that would be a little embarrassing but we've also seen API keys and version control before so I don't know if anything surprises us anymore yes that's a definitely a whole another challenge now one more thing I want to talk about though is the image generation I've left a couple links to that and also if you look up Nova Spirit Tech I think I have his video in there as well he's got a really good demo on YouTube of this the image generation is going to really rely if you want it to be reasonably fast at all on your GPU so you can run this locally and I actually thought about setting up Docker on my local computer because I got a fast GPU I used for mostly rendering videos but when it's not rendering videos I could definitely create some images with it there's some really fascinating tools that get you going with stable diffusion in a very automated way to create all the images you want and it's wild how well it works it's something that if this wasn't a podcast it's best demonstrated if I set one up I'm definitely going to do a demo on it because the demo given by Nova Spirit Tech on the generative images just kind of blew me away with how smooth it was now one of my staff actually set this up at work and was playing with it I was impressed they were training it I think they tried to make me look like a Pokemon because they were training it on images of me and they combined images of a Pokemon it was kind of horrific oh my gosh that's but it was funny that's pretty funny yeah that's kind of the nice thing is if you're wondering and you're looking for a project that might get your interest in HomeLab or just having these conversations with it it's pretty cool there's also another project I love to link to and I'm a fan of some of his work Daniel Messler he runs Unsupervised Learning it's a good podcast he talks about a lot of different things and I think he might get sometimes a little overhyped on some of the AI things but on the other side I think his dedication to open source is top-notch so I've left a link to his new project he's been slowly dumping on the internet called Fabric and it's something he's been building for a long time and he's wanted to put it all open source and once again it's one of the things like because it's so customized to be personal to him he's depersonalizing it much like Jay did with his installer to bring that more public for you know you know how challenging that can be you're like wait someone doesn't always want a user Jay created on every computer you run this on so but the Fabric one of the things I really like about it and if you just looked at it as an open source project that you want to look at the prompts used it's a way that once again you chain everything together from the command line he's using a lot of chat gpt but a lot of this once again can be pointed internally but where it's really cool is the prompt engineering that it does he's even got one that looks at and summarizes security problems into a JSON format that he loads into his own database so he has his own assessment of them and he's all got that broke down on there and I thought that was a really cool feature he did an interview with David Bonbol on it and there's a whole video I think David sometimes I really like David I think he made that video a little bit you know to click baby versus very technical and just getting the GitHub project and seeing how it works is much more technical and understanding the code and how the code comes together and how it prompts it so it's once again another cool thing to play with that kind of gets you moving on this and open source and it's completely self-hostable Self-hosted AI that's incredible like the conversations we can have nowadays in tech I mean just years ago it's just a different conversation it's pretty cool Yeah I that's what really surprised me you think of things like ChatGPT you know because that's the one that really brought it to light and how these models work as oh no these companies you're going to lock up this type of fun in some large corporate companies that's going to only charge me a subscription and subscriptions no matter how cheap HardBird gets over time subscriptions seem to go the exact opposite way they seem to go up and to the right all the time it's oh no you know HardBird got cheaper HardJay's got cheaper it's never been cheaper to buy all the storage oh cool so my bill's going down oh no no it's going up by 30% this year yeah that's it is yeah yeah when it's funny because that's a good topic to bring up Wendell loves talking about that so when he comes on our show he's going to be on very soon so um we have a number of guests and he is going to be and I'm looking at my calendar right now we have him scheduled for the very next episode we'll have Wendell on so that's pretty freaking cool and then of course we're not done yet because we also have Veronica from Veronica explains she's going to be on and I can't wait for that Veronica is so much fun she kills it with the retro stuff definitely love it it's always fun to have her on that that's really awesome and then on the sixth we have and this is this little pending right now the timing but it looks like we'll have techno tim on the podcast on the sixth so three episodes in a row we're going to have a guest it's going to be so much fun yeah we are we are loving all the different homelab I think this is really we're seeing such an uptick in homelab the accessibility of the hardware you know Wendell's been such a push for this I mean look at even 45 drives you got a big enterprise company that solves big problems and you know they're not going we're just ignoring the community you're like how do we engage with the community that makes me happy I accept a big announcement today they're doing better on some dedupe stuff once again Chris Moore engages with the community I really like this even though enterprise maybe where they make their money they're bringing it back down they they're showing some love to the homelab people so we're we're excited to have all these other people on there I think this is the it feels good being able to being able to host all this being able to do all these things without just constant reliance on oh no you have to run this in in AWS and cloud infrastructure and proprietaryness yeah proprietaryness got to watch out for that yeah so those are my thoughts on the AI stuff I like said it's a podcast so it's harder to go real in depth on these because some are going to be very visual and walking through things but I've left a lot of links in the show notes for you this will send you down a rabbit hole and keep you busy till our next episode for sure because there's a there's a lot of learning to do and even for me it's it's moving fast and it's been really fascinating to play with all these so absolutely yeah we love it this is this is uh this is how we learn this is how we pass our time this is how we appreciate technology and we love seeing all of you get involved in this and uh that's about it yep oh as always we should throw I'll throw this up here at the end because we always say send us feedback at the homelab.show we love hearing from you we like doing Q&A episodes so send us some feedback send us your thoughts and comments and all that fun things or leave them in the comments if you're watching this on YouTube we try to respond to all those as well yep absolutely all right and thanks