 Welcome to the homelab show episode 91. We're going to talk about CICD pipelines with Jane and among a few other things smaller topics that I'll bring up as well. Yeah, sometimes we're doing these shows where we're going to combine a few different smaller topics because there's just so many things to cover and this is this is a one where I haven't built software from a professional standpoint in about it's actually been 20 years. Because last time I did stuff was 2002 and 2001 working in corporate. So C++ I think or turbo C or oh much worse. We were using a programming language called progress and oddly I did some consulting with someone who's still using progress as a language. I'm like, wow, we're making progress with progress. They're making progress with it. I loved all the puns. We could play around with that. So my knowledge of this is a bit dated. Okay, a lot dated 20 years is a long time and there's there's been a lot of cool tools. Jay is familiar with these tools so he's going to be leading the discussion on this. Yeah, that's, that's fine. Well, it'll be a progressive discussion. Ah, yes. Now, good news is this is something before anyone wonders that is homelab relevant because many of you are working towards or heavy job doing some level of development whether you're a developer or you interact with them. I think this is pretty relevant is why we chose it as a topic and it's also just good to think about workflows you have for yourself because something that gets homelab people in trouble is not having documented workflows for things. So if you took an abstracted view of this from just documenting and thinking about how you work towards adding something to your homelab, this can be very helpful for that. Yeah, and also there's a very homelab specific implementation that is going to be at the center of it. So what I'll what I think I'll do is kind of explain in general what it all is and then we'll kind of just put it on the homelab side of things and then we'll see how that can actually help people out. Yeah. So, um, we'll just jump right into the show today. I didn't prepare to add read so and we've done it enough this month. Yeah, we have quotas in case anyone's wondering that's how we keep the bills paid and the lights on here at the homelab show. So let's jump into the Well, what I was thinking is I'll just get through a couple smaller topics that I wanted to bring up. I have something kind of fun that people can try as well that I thought be kind of cool. And then we could jump into it. So the first thing I wanted to bring up was chat GPT. Um, I know bandwagon alert, right. But, um, Joe from my other podcast had the suggestion I don't remember if he mentioned it on the air or not. But he just gave me a little challenge or a little friendly nudges like just paste your SSHD config file into chat GPT and ask it how to further enhance your security. I thought, well, that's a very interesting thing to do. So I will, of course, put a disclaimer out here that do not trust your companies or personal security to chat GPT. I'm not telling you that it's so accurate that if chat GPT says you're secure, you're secure. I'm not saying that, but it could be a way of getting some additional feedback. Now you will get mixed results. That's chat GPT right now. You can ask it the same question twice and get different answers both times. That's just how it is. So don't take it as gospel, but it might be something to do. And also be careful when you're just throwing your config files out there on the internet. I'm not saying to have a free for all there either. But if you put it in chat GPT and ask it how you could further enhance your security, you might get some pointers or maybe not. Again, chat GPT. Now, on my end, it didn't really give me much because I've been going over my SSD config for a long time. And it told me to disable password authentication, which was already disabled. So there's a little margin for error there, but I thought it might be fun. If you put in a config file to chat GPT and ask it how to secure it, it'd be interesting to see what comes back. And if something interesting comes back, let us know in the feedback because I think that would be a great idea is just to see if you had any success with this, if it found anything that you didn't think of yourself. That would be pretty cool. Just let us know in the feedback. And by the way, feedback. Come on, guys, you need more feedback. So we'd like to ask questions feedback at the homelab show. So that's an email us feedback. We made it easy as we say an email. One more comment on chat GPT. I kind of like the joke of when you say chat GPT is smarter than a human, you do have to specify which human is smarter than. But that being said, it really does a good job on deterministic things better than usual. So code is one of those things that's less opinion and more accuracy. So I've actually found, especially since they moved to chat GPT for when you ask it, some of the code questions and my staff has been using it for this. It gives really good answers. They were dealing with it. I don't know if they solved it just yet, but I know they were talking about a timeout script. And I'm always like, guys, ask chat GPT a lot of this stuff. It can do this. I've had it write several things for me because I'm like, wow, this just does it. And one of the things it will note is when you tell it to do something, I even tried to get it to do something insecure. I was, I thought I was just going to make a video dumping on chat GPT. You know, there's a reason that video doesn't exist because it won the argument with me. It would not easily let me. You really had to try to make it do something insecure. I was trying to write some bad firewall rules and things like that. But it's actually very good. It even gives suggestions for firewalls or how to set something up. We were actually shocked at the accuracy by which it could give you some advanced instructions on how to set something up in PF sense. So I think it's a great tool for the homelab people because if you are one of the people who are just getting started in this and you need some help staring at a blank sheet of paper is hard. Getting that first form written for the code is hard. I'll have chat to build you a Python script to do the thing you're asking it. And even if it's not 100% correct, if it's 90% there, you're 90% further than you were before you asked the question. So I'm still bullish on using it, especially in a homelab and learning environments because it also makes notes. It does better than senior developers do at putting comments and code. I'm telling you what the code does, which is great for learning. Bad for senior developers. It really is. It's a fun tool and I'm interested to see where it goes. You know, it's going to be our overlord at some point. We all know that, right? I'm just trying to be on good terms with it ahead of time. Of course, I don't actually believe that, but it is kind of fun to play around with that sci-fi thing. I just don't want to get to a point where I'm on there and I'm asking it something and it says, I can't do that, Jay. Sci-fi reference with a name swap on that one, but anyway, just thought I'd mention that is a fun thing that might be within reason. Obviously, I don't know why you'd have anything proprietary in your config file. You shouldn't, but just be mindful of that and maybe you might get some pointers or maybe you might get some nonsense. Either way, it's fun. Yep. All right. What's the next line item here? Line item. I just wanted to mention Cubic really quick. I had some time to play with this. And this is how I created the Ubuntu Flatpak remix recently was with this tool, which allows you to grab an Ubuntu ISO image and just do whatever you want. You could just choose different packages that are in there by default. Basically make your own custom version of an Ubuntu ISO. So if you wanted to create an Ubuntu ISO for whatever reason in your home lab, it might be a tool that you could use for that. It's pretty fun, actually. If nothing else, it's pretty easy to use. It's like a little wizard. You just keep clicking next and just tell it what you want. And what's really cool is it'll actually allow you to test your distro in a new window that comes up. You can actually play with your distro before you finalize it and then create the ISO image. So I think it's just so cool and I think people should check it out. I'm not saying everyone should start their own distribution, but if you wanted to just... There's like a couple of things you do with an Ubuntu release, remove this, add that, whatever it is. You could just have that right in the ISO and just not have to do that when you go to deploy it on an instance. So just thought I'd throw that out there. It's called Cubic. It's available on GitHub and it's a lot of fun. We'll throw a link to it in the show notes. And how long until it replaces, Jay, your Ansible poll that you're doing? It can't. And the reason why is because it's... The most I would do is implement Ansible into it because it doesn't do full configuration management. I mean, you probably could, but re-spinning a new ISO every single time you make changes to your Ansible would be really hard to do. And I think that we'll come back to that thought, actually, because I think that's been thought to explore the CICD thing because it kind of goes in there, plays pretty well with that. So the last subtopic to bring up, which is also going to have a question to the audience as well, but I just wanted to mention this for those of you that haven't heard of it before. It's called Barrier. I use it in the office in the studio here. That's how I record or I manage the recording PC and my desktop with the same keyboard and mouse. So basically, I could just be here. I could be there. I just move the mouse off the screen and then it's on the other screen. And I'm controlling my other computer. Then I just move it over and I'm controlling a different machine. So it's basically like network or networking your keyboard and mouse between two different computers, which is really cool. Right now it's not working. So I have this little small keyboard here that I'm trying to use just in case I need to control the other computer. Because the issue, and this is the question I was going to bring up to the audience, if there's a easy solution to this, is what do you do if you have more than one computer? I have my main desktop that is able to control my other desktop, which is the recording computer. But what if I'm using my laptop, for example? Okay, it's really hard as far as I've read to have two computers sharing a keyboard and mouse. So there's got to be an easy solution to this. And I have to be over a complicating this. So the idea being, what if I have my laptop plugged in and I want to control the same computer as a laptop that I also control the desktop using something like barrier. But obviously, most people aren't going to have a complicated need like I do. So I think for most people, barrier by default is just a good recommendation to the audience if you didn't already know that it exists. I'm trying to remember what it was called before it switched to barrier. It wasn't, was it Synapse? No. It had a different name before, but yeah. Something like Synapse or it wasn't Synaptic because that's the apt tool on Debian. Something like that. I know someone in the audience will definitely remember it. I just want to remember its original name. By the way, if you're looking for a video on how to use barrier Veronica explains, good friend of the channel. She's got a great video on that topic. But we're looking for some solutions from the brain trust of our audience here. And we'll talk about it in the next episode. But yeah, different look at what barrier is. If you have a solution that works the same as barrier or tips and tricks for using barrier. We're all ears on that. I'm interested as well. So, we're going to get into the CI CD topic and I want to prefix this by saying it's not going to be a deep dive. I'm going to talk about everything I know at the top of my head, obviously, but we could go deep into individual solutions in future episodes. This is going to be more of a foundational thing just to kind of first talk about what it is, which is going to give everyone the question. Well, what the heck does this have to do with me if I'm not a software engineer, but we'll get to that. So continuous integration, continuous development to really simplify this. I mean, just imagine you have a team of software engineers and they're, you know, basically releasing software at some point, compiling, testing, compiling, testing that can be a little difficult and time consuming. So one of the ideas that you have build agents. So when you do a git push, for example, or you merge something, then you could have something compile that and email you, you know, email the developer or whoever it is that's managing the project. It was successful. It was a failure, whatever it was. So you fix your code, push it again, failure, fix it, push it again, keep repeating and eventually, you know, everything's green and everything's working. I worked in a development team where we had like, I want to say 12 build agents on a Team City server, Team City being one of the solutions, at least back then I haven't looked into it recently, but these build agents would constantly be building code because this company was always releasing new code. It was great to see this. And if there's any problems, then before the software got, you know, released, it would be caught because it's being built, you know, in tested beforehand. Obviously, there's a lot more than just continuous integration, continuous development in their pipeline, but that's just the subject we're talking about today. So one of the popular solutions is Jenkins, which is basically my go to its open source. It's easy to use. It's a little weird at first to kind of wrap your head around this style because, you know, you have to set up a build agent. It's not hard to do. It's just different, you know, and it also has a built in build agent, which is a little interesting to me because I always think of the server as the server and the agents as the agent having an agent on the server is weird, but that's another story. But basically you could set up agents to do whatever you want when it comes to building things. But I mean, that's a gross oversimplification of what CI CD is. It's a very powerful tool in the enterprise for software engineering and is probably very transformative as well. But one of the topics that we need to get into is what does this have to do with home lab because, you know, am I trying to say everyone needs to start writing CC plus plus and Python and you actually could because it's a lot of fun but that's not for everyone. So bringing it back to home lab is often the case I think where you run into a situation where the work of maintaining a home lab or your configuration management code could be work in and of itself and at what point are you spending more time managing your Ansible chef, you know, puppet whatever it is, and you're spending more time doing that than you are enjoying your home lab and then you're basically doing the work of a software company at that point. Sure, you're not necessarily developing software unless you are, but you're still writing code. I mean configuration management is code, especially considering Ruby is used in a lot of these. So it's code is code. It's it's code. And it does kind of cross that line at some point in my opinion and I'll use myself as an example. I think I did use this example for something in the past so some people that have been listening probably already know this but when it comes to my Ansible config it's basically it does everything work stations servers it knows how how to set up a workstation and it's great it's completely hands off. But the problem started when, you know, it just keeps getting bigger and bigger and bigger and harder to manage. So what do I do and what makes it even harder is I support Arch Linux, Debian, Ubuntu, Pop OS and I'm probably going to add Fedora to it as well here pretty soon. So then it creates a situation where every time I implement something into Ansible, I need to know if it's going to work on Debian is it going to work on Ubuntu is it going to work on Fedora that's kind of annoying because I have to test it that many times. And that creates a situation where every, you know, every change I make to Ansible becomes a lot of work because it has to pass each of those tests. So what I decided to do was create my own CI CD pipeline but I kind of hesitate to call it that because I'm not using Jenkins in production yet. I probably will, and I'll come back to Jenkins later, but it just is very very basic. So what I've done is I've created a reference VM, a vanilla reference VM for each of the distributions that I support, and I create a branch called staging, and then I, you know, basically, I develop against that branch. And these instances, check that branch out everything else will check out the main branch that's the primary one that production gets. But when I'm developing something, I just develop against staging and since these instances check out staging because Ansible pull is local, not, you know, the other way around to the server, then they'll run it against themselves. And I have something called pushover for notifications. If it's a success or a failure, it'll let me know on my phone, which is pretty cool. Every time I submit something via Ansible or I push a change that branch, I get a notification on my phone from each one of these servers that a pastor failed and then I could look at the failure because the message is right on my phone. What the heck happened and there's the error message. Oh yeah, that's right. I had an extra pipe symbol and this actually happened yesterday. And I didn't catch this myself, but it failed the build and I figured out that's why from the error message, delete it, push it again, then that server comes back and says that it's successful. And at that point, I have a system where I just simply push out something to my Ansible config or my repository. And then each of these reference VMs will let me know how Ansible did on those machines. And that saves me from pushing something to production and having 30 machines emailing me complaining that something's not working. I'd rather it be somewhere between one and four test machines and everything complaining and things falling over, which is just going to be a dreadful thing to manage. So I've got to hydrate. No, it's it's really interesting. It's all the testing that goes in there and this is makes or breaks your software or your deployments because we can't just deploy everything in production, especially once you get bigger. You have to really schedule so you want to do as much testing ahead of time to be is just not disruptive as possible. Yeah, exactly. And I have a feeling that there's at least one person that's going to our feedback email right now and they're probably writing something along the lines of, Jay, you wouldn't need all that if you just like settle on one distribution, then it wouldn't matter. And that for if someone's going to say that they're probably right but my mentality about Linux distributions is that you should never have just one you can have a primary. You should always have at least one secondary because your primary distribution could change at some point to where you don't want to use it anymore because the company or if it's a company behind it might decide something that doesn't go along with what you want, for example, CentOS and they're moved to CentOS stream. I have a feeling if I was to ask people before then if CentOS as they knew it was going to go away they say no it's my only distro that's all I use and that's all I'm ever going to use and after that announcement Oh, maybe I should switch. And then when it comes to companies they, you know, have a hard time switching between distributions it's it's not an easy thing to do. But if you're always maintaining reference configurations for at least one secondary. I could change my main distro, which currently is a boon to server on the servers and then pop us on the desktop. I could change that tomorrow if either one of those goes the direction I don't want. And I maintain those configs and it especially helps with the channel because if I want to have an arch Linux config set up to go over that in a video then my answer will take care of that for me and I can easily do that so there is value in, you know, one other distribution but then again, if you're, you know, using another distribution is your plan B and you're using the same Ansible config, you're going to run into package names being different you're going to run into config files, maybe being named differently really depends each distributions different there. So you have to test variables for one distribution versus another and it just gets complicated but having CI CD in the home lab is, I think a great way to tackle this, but it goes even further, because I haven't done this myself, but I've seen people use, no actually I have done this. See, I make so many videos I can't remember how many I actually do. I knew there's a reason I was thinking of this I did it for the nodes channel that's why, and I guess that's a spoiler if you didn't know that video was coming. But essentially what it does is it sets up Jenkins with Terraform to build VMs through Jenkins. So imagine having a configuration with a with the API for your cloud provider if you have one or even proxmox if you don't. I imagine other virtualization solutions have an API as well. So imagine spinning up virtual machines by just going on to your Jenkins server, clicking one button, and then you just get an email servers ready here's the IP. You could absolutely do that with CI CD and you could argue that's not necessarily CI CD you're using Jenkins is more of a delivery service but these tools and that's why they're kind of hard to explain they do more than just build your stuff. They literally just execute, you know, commands or something or even use Jenkins as a basic ansible server if you wanted to. But I figured it was probably a good idea just to kind of put this out there and get this idea into people's mindsets especially if we explore these topics in deeper mentality later on which you know probably will, especially as I get more into Jenkins which you know how that goes I'll put a video on it video series or maybe even a series that's kind of how it goes. But these tools that are normally made for software engineers if you just think about it a different way. It might be useful for us to where clever people that's why we have a homelab right because we're all very clever so finding clever use of these tools is kind of the name of the game and CI CD might be a good way to automate some of the complete server builds if that's something that you, you know, might solve your use case. And I think it's always a goal to work towards, because I'm going to work on soon, the more basic versions of doing this and it's, you got to start somewhere. And I obviously am a big fan of XP and G, but a lot of people get confused on how the templates work and how you build things and sometimes people get a little lost on it. One of the things that having this allows me to iterate fast on matter of fact, I have a friend who's been doing it in a more complicated way. He looked at mine and he goes, Oh, that's kind of simple. So there's other ways to do it, but it's all having that starting point because you want to be able to generate VMs fast. You want to be able to deploy fast to be able to get to the thing you wanted to do. I was doing a video just the other day on Artwatch. I needed to create several different VMs and I also needed to change the MAC address on those VMs. So it's like looking at ways to make that simpler when you do it. If you didn't watch that video, if you change some of the prefixes, I was doing some Mac spoofing for fun. You know what you do for fun, right? I mean, MAC addresses. Right. Well, it was because we are identifying things by their MAC address and what they were. So all of a sudden I had Lockheed Martin tactical systems showing up in my, in my look, you've got, would you like to know if these things have been added? You know, this is, I think I want to re-change the Mac on my laptop. So everyone thinks that Lockheed is now attached to their network because that just seems fun. Wait, does there a military company attached? Oh my gosh. The level of pranking. I'm not trying to be an enabler here. So I should shut my mouth because I'll get myself in trouble. But yeah, that's, I mean, those are clever use cases as well. I think, you know, just finding good use cases for this stuff is really fun. I wanted to mention someone in the chat room. Let's see if it. Oh, Jaden mentioned its synergy. That's right. That was the original name for barrier. They kind of forked over to barrier at some point and barrier is kind of hard to find sometimes when you Google for it because it's such a generic word. You could find like a barrier for something or the magic spell and the Final Fantasy series, which is like an alternative to the wall spell. You have the barrier spell and then you have barrier for your keyboard and mouse among many other uses for that word. So if it were me, I would have chose a more standout word for that project, but it is a good one nonetheless. If you could find it, you just search barrier keyboard and mouse and we'll have a link in the description for this as well. Want to check that out. It's also the same problem with there's always name collisions because apparently we can't come up with new words. So when we write new languages, we go, Hey, Rust, this is a great name for a language, right? No, you know how hard that was until it became popular. Yeah. And music fans wonder why churches spelled the name of their band with a V instead of a U and they literally said it's because it stands out on Google. That's why we did that. Yes. And businesses do that all the time. But Rust has gotten popular enough to where I think it's, you know, not going to be a problem for Rust at all. Yeah. I see a common question here from Steven. I know he does some software with all these comments on channel before here. I've been working on a GitLab CI runner running to my OpenShift cluster this morning. So how does that work, Jay? Is this a good integration going with GitLab too? Because you just did a bunch of, oh, you did OpenStack. But didn't you look at it as well? Well, so basically, yeah, I mean, that is something that could be done. If you think about it in the video, I walked through the process of, for example, setting up OpenShift on OpenStack, which was a lot of fun to produce. Let me tell you, I had so many errors. It was like, I think a three hour recording at least for a one hour video when I edited it down because there's, you know, people think, oh, you always get it right the first time. That's what you see. But behind the camera, there's F-bombs and there's, oh my God, why don't you work? And then once I figure it out, of course, all that gets edited out and you see like all the successes. But anyway, the point is I did that manually and I go through all the commands. But any of those commands, you could script and have something like, you know, GitLab, for example, that by the way, that has CI CD built into it as well. Git Hub does as well. So there's solutions right in your repository managers for this too. So absolutely, that's something that you could do because there's an API so you could hit the API. I want an OpenShift cluster. I want an OpenStack instance set up or an OpenStack user security group or something like that. I would personally probably use Terraform as the bridge in between the two because Terraform, you could test it against OpenStack and know that it works. And once you have that down, you put your Terraform configs in a repository, put that repository in the Jenkins or whatever it is you use, didn't know where that repository is, you know, pull down this repository, watch this repository when there's a change, or I give you the instruction to do your thing, you know, activate the script and use Terraform spin open instance for me. And that's useful for home because you want to just test out a VM randomly or even at work if you use it at work because how many times if you do this for a job, do you have someone at your desk that, can you build me a VM? Can you build me a VM? And pretty much you're just building VMs all day. At that point, you probably start automating it. Automation. It goes outside the topic of the Homelapse show because this is completely Tom and Jay problems. That's what we were discussing beforehand is automation doing build pipelines for creating content. Yeah, I mean, that is something that we talk about too. It's the same though, we have a workflow that we have to go through and we're trying to, you know, put that pipeline in place and make it as automated as possible. But some of the tools aren't there yet. The people who make video editing and audio software are not at the same level of people who automated the Jenkins build system. There's no Jenkins version that we found yet that's quite as automated for the video production side. Yeah, so we're having random conversations like I wonder if ProRes, if that's something I could record over one gig, even though I'm editing it in 10 gig, you know, that those, those questions kind of come up behind the scenes. And sometimes I wonder how many people care about that because how many people has a problem that something like that would solve. I wouldn't think very many, but maybe I'm wrong. Yeah, I mean, you know, maybe me and Jay will co-opt a video on our workflow, not necessarily because you're a video editor and it's interesting, but maybe the tools, the true NASA's we have, how we've got them all connected, how we have multiple computers sending to the NASA. And this could also be substituted out for things like a Synology. But it's thinking about how you get that data from where it is because maybe you work as an IT administrator. And you're like, hey, I wonder how I can help the teams that do the media on the other side. So there might be some value in it, not as a homelab show, but just something me and Jay do on our channels. So, but that goes back to feedback at the homelab show. You can leave comments in this video. You can send feedback at the homelab show. Either one of those options will bring that data to us and let us know what you think of it. Yep. Yeah, absolutely. We want to do another Q&A and our ability to do so depends on you. Yeah, we like answering your questions. It's a lot of fun. Yeah, we like it a lot. We love when we had them ahead of time because that's when we get more in-depth answers because we have time to actually research and put things together. Maybe one day we should just set up ahead of time a Q&A live Q&A clinic or computer homelab clinic or something and just bring all your stuff and let people know ahead of time. Yeah, we're here to solve your homelab issues. All right. Is there anything else we have, Jay? I could go all day, but I think that's our topic. That's our topic for today. We kept it simple. We kept it short for you. Looking forward to hearing from everyone and we'll join you next week. Thanks. Yep. Thank you.