 Welcome to the homelab show episode 25 version control with get and I'm gonna try to make a lot of puns in this because some people may not get it some people get it like Jay and Get off my lawn. Yeah, if we can just do play on word with this I'm going to explain all day. I gotta get a few of them out as soon as we start here But this is an important topic because get is well when people ask I hate Tom How'd you get your prompt like that? Well here just get cloned and I have a little installed it installs the The dot files so you can have this in Linux and things like that It's an easy way for us to and Jay has a github, but don't conflate believe it will clear us up right away Get hub uses get github is not get they're not interchangeable. Get hub is a service owned by The evil Empire that is Microsoft. Sorry. I have to call him that Yeah, I'm glad you pointed that out. That's a common misconception. Yeah, so there's I really will clear that up right away It is a great place. I myself completely do use get I think Github is not a bad place necessary to host it But beware that is owned by Microsoft and get is a process get hub is just a place to put things because there's also get lab But Microsoft didn't buy them in an honest popular Yeah, and I'll talk about a little bit about get lab I'm not an expert, but I will talk a little bit about that and another solution that you didn't mention that Is not the first thing that comes to mind that I think it is worth mentioning And I might even do a video on it if enough people want me to but we'll get to that. Yeah, we'll get to that See, yeah, we'll get to that Before we get to that we will talk about the sponsor of this show and that's the node And you know what that might be a fun idea you could probably run your own get system in Linode Yeah, you could get your own virtual server and run it there. Yeah And you know like you won't turn this podcast off before Not yeah, it's getting a little dry now, but honestly, you know getting it back to Non-get related top actually we should keep it to get related topics because like I said you can run your get server there but the node itself is the Sponsor of this episode like most of the other ones so far which we're really happy to have them on board They host the podcast for us. So you are using you're actually using Linode if you go to the website for this channel download the podcast Yes, and it's a it's a fantastic service all of the web presence for learn Linux TV runs on that as well and Couldn't be happier with it. Yes. So we've been Linode users for a long time and So we like to think them as a sponsor is an offer code down below so you can get in on it, too So Linode's been pretty awesome. All right back to the topic at hand Yeah version control So let's so I figured for this we could kind of You know, well, we'll just kind of walk our way up through the the topics and just kind of start small and you know Easy and then work our way up. Um, this isn't going to be a tutorial. This is a podcast But I am going to be mentioning a few things about get and you know be the command line as part of that But first of all get is part of version control and version control is where you want to control versions essentially You want to be able to keep track of changes and it's often a thing that's used for software developers And in fact, that's where the primary use for version control is and this isn't a software development podcast So why are we talking about that? Well, because version control people have realized is a concept that Goes beyond just software development and you can use it in ways that may not actually be apparent at first to most if you're thinking about becoming a software engineer you you pretty much learn version control and I remember when I was taking college classes before I graduated I was really upset that there wasn't any classes on version control Even when my major used to be at one time So for software engineering, I'm thinking this is a big deal, but when it comes to home lab and Business you could keep you could keep track of different versions of things other than you know You're C++ programs or whatever it is you're writing and you could keep track of your config files Your server config files your dot files and I also feel that for a lot of people version control is a precursor to You know getting started with automation and that's kind of how it was for me like when I first started with Linux I discovered bash scripting one of the first things and I remembered that there's all these commands any time I set up a distribution that I have to run every time So if I wipe it out reloaded have to run these and I would just You know just paste these commands into a text file So if I ever wanted to set up another distro I would just copy and paste the commands into a terminal to set it up But then when I discovered bash scripting it's like oh, I could just make this a bash script I have all the commands here So I'll just make it executable put the bin bash header at the top of it and just be able to run it I could just copy that file to the machine. So this is before I got into automation This was kind of like a form of automation It starts like this for most people and then it's like well, where do you put it? Well, I'll put it in version control. So that way I have a place to download it from So get is just is one of many different types. I mean there's Subversion or SVN for short for example Mercurial a number of other ones But get has just become the most popular. So that's what we're going to be talking about. Yeah, and SVN I believe was like Before get SVM was kind of like the de facto with the majority of people used it was But not it just didn't quite work quite as well as get does Yeah, I think it has a lot of Things that makes it more popular that I've read but full disclaimer. I pretty much started with get I did Briefly use SVN for a few months, but not long enough to where I could really just give you like a point-by-point comparison to So why is get important in this space? So if you think about Servers you think about the fact that things change over time So you have a let's just say you have an Apache config file that has your You know server controls in there and then you you know change it up or maybe you make a template or something Where do you store that so other people can get to it and also if you have it under version control? Which we'll talk about later if the server is not acting right, okay What changed but with get you can actually go back and find out like okay? Is this config file difference since the last time it was committed now taking a step all the way back What's required for get you mentioned earlier and I'm glad you did that Get and get hub are not one in the same get hub is like a web wrapper around get you could use get on The command line you don't need a GUI for it. In fact and a lot of people may not realize this You don't even need a server. So let's just start at that level Let's just say you have a Linux laptop or a Linux server You have no it's not a server in the sense that it's not it's not a get server It's just maybe your web server your Plex server something local that isn't accessible from the outside So if you install get which would be just apt install GIT get or dnf install whatever your flavor is then you could use get right then and there and the code never has to leave Your system it could stay there on the system and never touch the internet never be uploaded anywhere So if you for example just creating new Repository then you have a folder and anything you put in that folder you can add it to version control Which means you make it care about it get is going to pay attention to it Because you put a file in a folder does it mean that get is all of a sudden going to be oh I'm going to keep track of that now. No, it's it'll basically be a foreign object But you Get add that particular file and that tells get you need to keep track of this file at this point Nothing has left your computer your server yet. It's just there on your local file system now Why would that be important so? You can also initialize a an existing directory So say for example, you have your a pat your Etsy Apache 2 or Etsy HTTBD folder for your web server So you could just get in it inside that folder and now that folder Has a dot get folder inside of it and now you can actually Manage the changes in there and add your Apache config files to version control at this point Nothing has left your server yet. It's still there. It's not online. It's not accessible But what this allows you to do is if like something changes or something's not acting, right? You can run a get status and it'll call out anything that has changed since the previous commit You commit every time you make changes. This means like I want this is a final change I want you to keep track of this and that's a great thing to do because if you have other people in your house Maybe you and a friend or I don't know family member also get into the server Something doesn't work. You could say hey, what are you doing over here? You don't you totally deleted this line right here and you know that because it's you know It's under version control and kid is taking care of that and watching that for you Well, there's also when you run updates Occasionally, I mean granted the update does ask generally speaking The update itself can make changes to a config file because sometimes when they update a new version and then get can Allow you to see the differential changes between there so you can it's telling you but then you know We're next in yes things sometimes through the updates and you're like, oh, yeah I yes about three different changes now I should probably look at a differential between what it looks like now in my last commit And it's you know, just really is that version control way of tracking to see what's different It's also really handy when you're iterating through configurations to Figure out what you did each step of the way and being able to look at different timelines or even re-roll back to that particular one When I was configuring you know bring up media wiki because it can be tedious to configure I was not smart I did not use get and sometimes I'm like, oh I by fixing something over here something broke But I remember what it is that I did because now this function is broke to get this function to work and I if not having a good differential made that hard absolutely and It's super easy to just install get on your home lab server Go into a folder that has a config file as you care about and just initialize a Repository there again It doesn't have to leave your computer if you're concerned about privacy or something like that Especially considering some things and this is important. We'll get to talk more about this something should never be in version control It's I mean that that's something we really need to dial down here because there's some things And I'm going to keep pushing this and mentioning this over and over again until people are tired of hearing about it Be very careful what you put in there because it's hard if it does reach the internet You do upload it to a server a get server then it's out. It's out there So anyway, you could control your servers with it and if you have a home lab server There's probably no reason not to initialize a repository inside the directories that contain the config files that are important to you Now another thing that people do is they take they want to keep track of their dot files They're Vim RC their bash RC their Z shell config files Tmux config files all these different things and I think that's how a lot of people get started with version control because You know sure you could just grab those files from your home directory Put them in a zip file or a tar file and just you know manually copy it over and extract it You could do that but I think it's it's pretty cool to keep track of your of your dot files And there's also a I haven't used it myself But there's a tool called stow stow that a lot of other people really like for this purpose So it could keep track of your dot files it in I don't know if it works this way But I know when I used to use get for this purpose I would have a dot dot files directory that's hidden and inside there be all my config files in one folder And then I would have like a knit dot sh or something where you just run this can this bash script into just sim links You know dot bash RC to dot dot file slash bash RC or wherever it is So just put sim links where the files are supposed to be so your shell can find them But they're in this one folder and I think stow might do something similar to automate this I'm not exactly sure again. I haven't used it I just know like I wanted to give it a mention because a lot of people love it so I do it with the sim link method that you mentioned that's how I started with mine get because the Dot files is something you know I show them a lot in my Linux tutorial videos and things like that and people like How'd you get the command line customized like that and I have my github because it's one of the easiest places to throw them for public Accessibility and it's also how whenever I set up new servers It's the same thing I like to put in there and using a version control from my main computer Means if I change something in T-Mux or I change a feature in Vim RC or bash RC to the way I like it any subsequent servers deployed later or I can just hit get pull on all the other ones and have them Do that and to reiterate what Jay said Please note one of the good reasons for doing it in a folder in a sim link as opposed to just your home folder is you don't not Watch your SSH private keys over on github. That is bad. That is something that happens a lot And there was a time we're in github for API keys that people accidentally uploaded and especially they work for a company That's especially bad like the AWS keys like oh my gosh You could think about that like if you if you work with AWS or any of the other VPS or cloud providers and you have your keys your admin keys that allow you to launch machines and remove infrastructure That if that goes up into a server like github or an online resource. It's a bad thing I I say get used to not including those things even if you have no intention ever of the you know get Repository ever leaving your computer. Maybe you don't want to host it online at least make sure you don't include files in there And how do you do that? How do you ignore a file? You have it Let's just say you have a file in there You want all of the files in version control, but you don't want this one though because that has like some maybe an API key in there You don't want that so what you could do is you could create a dot get ignore file And you put the file name in there that you want it to ignore and it's not going to It's just not going to keep track of that now if you've already added it on accident you have to remove it So if you've added it to version control Get ignore is not going to work the way you think it is it's not going to delete it You know on the version control if you have already entered it in there Just create a dot get ignore file and make sure that the files you that are sensitive are not Going to be uploaded by putting them in that file and you commit the dot get ignore file itself to make it take Effect and at that point get is like oh you've changed that file I don't care because like I'm not even watching that or that's a new file But I don't care about it and you get in the habit before you even go up to a server. That's very important. So Now we get into the topic of a server Get hub is one you mentioned get lab as well. Yep, but what's required to make a server and it's actually extremely simple You can have a remote get server by simply installing the get package on that remote server So say for example, it's a Linode instance, right? You you don't have to have a GUI you don't have to have a web browser attached to it a patchy none of that If you have just SSH to that server and get on the other side, you can actually push changes to that server You don't actually need get hub You don't need get lab if all you're trying to do is have your your code on a different server SSH done you can actually use SSH to push and pull from a get repository So then you have that Folder that you want to you know keep track of you have your dot get ignore in there So it's not keep you track of things You don't want to it's time to put it up on the internet So or a server that you control and you do that with SSH and that works Another thing you can do is if you think about like Tom's videos with zero tier You could set up a Linode server that is not publicly accessible That is external, which is great But you could make it so that the firewall only allows the IP for zero tier So it's not like anyone else can get to it So even though it's a cloud provider, you can still have a remote server That's yours and only you can access it. So that might be an option for some of you out there Get hub is another one of these it's the most famous you can you have your local repository You could push it right up to get hub you could push it up to get lab You could push it to all of them if you want. I mean, it doesn't matter Because that's the the get server isn't required the get server is just a place to store Your version control files and that's the important distinction Yeah, and get hub is one of the what made them popular is they did a good job of creating a collaborative community And a lot of a lot of big open source projects jumped on that And if you want to work with lots of people and you want to tie in a way for people to make pull requests I made a way for people to make Easily look at the public code and make comments on it So you can have some feedback from the public publicly posted on an open source project They did a nice job of putting that all together and it's what made them so popular along with just Everything was kind of well done when they assembled it. Get labs a good runner up for that But you'll I still see as many people as we're going to exit from get hub so far Even though microsoft bought a few years ago. I'll admit they haven't done anything Horrible to it right and in fact they added a couple features one of the features they've added is it looks for bucket keys and stuff like that It looks surprising You know, maybe the next few years i'll be saying something different for now It's nice and nice things when you're dealing with all your public code It's not like it's hard to copy all the code do a quick get clone pull something that you're looking for on there and One segue a little bit about that is this can be a problem One of the reasons if I find any type of project I like a github I always make a clone of it because sometimes package maintainers are Flighty and they just disappear code and I ran into something that was just missing the other day I'm like, huh I'm glad I have a copy of that because they have Plainly just deleted it off github because I don't know why uh, they're not in contact anymore They seem to have deleted their count Or sometimes even worse it could be something like youtube dl that got pulled down Yeah, yeah, and that's another example that wouldn't right there was a conflict That was resolved later about them pulling youtube download They always have a tenuous relationship with people who post security research on github They want to make sure that before security researchers post proof of concepts This is microsoft stands on it that there's been patches for it But sometimes there's even been a patch for it and the security researchers have found themselves kind of stuck in band from Uh github temporarily from posting some of those things That's just a little bit of it. So if you find something like on github and you have the space to store it add it to your own It's not bad if it's an important tool that you use all the time to have your own local copy that you keep up to date Yeah, absolutely. That's that's a great point and that's easy to automate Yeah, it really is and I think that uh get is one of those things that I recommend that literally everybody learn it And there's a few things I could say that about because most of these things It's very context sensitive like if if you if you don't have a use case for something What's the point in using it? But get is very important for I think everyone because even with homelab You're going to find code on github a script Maybe that does something you want to do and you're going to look at the script and Download it and use it. Maybe you might notice a problem with it And then you just say I'll just submit a poll request to you know fix this typo or something that I found in the script To kind of give back But one thing that I think the easiest way to learn it in my opinion for a lot of people is Especially when you're new to to all of this You're writing down notes probably when you learn something you watch your videos You watch you know watch my videos and then you're writing down notes now You have if you're like me I write notes and text files And then it becomes very easy to upload the text files as long as you don't include passwords or anything in there That's sensitive You know other people can benefit from that So if you just went through the vim tutorials and now you have all these notes about using vim You upload it. Maybe someone else might see that and they might oh, that's cool A little vim cheat sheet this person made while watching a video. That's pretty cool But as you create notes as you're learning all these concepts and pushing the notes up to Github and then maybe pulling the notes down on a different computer because you want to you know You can't remember the said syntax or something, but you wrote it down. You can just get clone it That's a really great way to learn version control Um, especially considering there's no systems hanging in the balance like you know bad commit isn't going to bring systems down It's just your notes and that's a great way to get into the workflow. I think um You know someone has an interesting question that I I think because gits been around for 15 years I remember people really hated Uh svm like it was a lot more clunky and harder to use compared to git But because git's been uh in use for such a long time It's kind of become Back to what we said earlier what it's become a standard. Also if anyone didn't know You might have heard of the guy that created git, you know his line is torvals It was a little project he had besides this other little project he has called linux He came up with that and it's It kind of became once linux and some of the big names started using it and from my understanding I'm not a developer full-time the development community says life was much worse prior to git Like it was a lot more difficult to manage larger scale projects. So it is um done that matter of fact Even microsoft's using if I'm not mistaken using git still internally now for a lot of their larger projects Which has probably led to some of their acquisition to get out there's it's been It's proven to be very scalable for a large project. So it kind of it solves a lot of problems in between Yeah, it does and um, I I don't remember much about svm. Unfortunately. I worked at a company That um a long time ago. They were using that as their primary version control But they're in the process of switching to git and they they like to get a lot better But they they started with svm. They've been using it for a very long time So it wasn't an easy thing for them. They had a lot of different build systems that were using svm Yeah, so they were they were making good progress and switching over to it. Um, and they They used svm. They just They didn't really complain a whole lot, but I could tell that they just felt like it is easier I mean, why else are they gonna, you know, move to it that it was a better solution for them there were a lot of situations where um, somehow some protection wasn't in place where It was allowed. You you were allowed to basically commit to the main repository But that wasn't what you're supposed to do. Um, and people did it and then repairing an svm repository Every time they had to do that was not a fun thing. They were it was a day long ordeal for them, unfortunately I could tell why they might want to switch to git. Yeah Linus created it because he needed to solve a problem. So he's like, okay I build the linux kernel. I got problems and it isn't I need to solve this problem I need a better way to build it So he took time off wrote git and went back to Using it for the linux kernel. So Yeah, absolutely. So I see a lot of questions in the live chat or statements about git lab versus github and vice versa. So Which one's better for the other? I think they're both great. I use git lab primarily But I also have a github repository for the youtube channel because that's you know, where all the cool kids are anyway Yeah, git lab kind of seems in my opinion to target the enterprise more Um, get it's not that github doesn't They do it's just they're more in my opinion a general purpose solution Whereas git lab is more targeted towards the enterprise. But however Um, home lab people can use git lab. You can install it on your own server Which is one thing I like about git lab. You could use their service And you know, they host for you or you could just host it yourself Yeah, and then you can't hold as far as I know you can't host your own github. No, that's the benefit and they have CI cd continuous integration continuous development. I believe it stands where I've been using that acronym for so long. I forgot the meaning or what it stands where I'm pretty sure that's right and They have that built in I think github does now as well But there's all these tools in git lab that I think are really cool. I recommend just checking it out You could just sign up for a free account there. They're not a sponsor or anything You know, just check them out see what you think of it Compared to github make your own determination which one you like best But um, you can mirror the two you can use both you could have commits from one go to the other So that way you could just have you know, I have to keep track of multiple remotes on that I like git lab a lot and I can't even remember why I chose it It's just I've been using it for so long now And it has some, you know, there's a lot of feature overlaps So it gets really hard to call one the superior of between the other It's just if your enterprise company focus git lab if you're You know general purpose developer or don't have any specific ties to the enterprise then github is probably fine But I think even if you use git lab You'll still find yourself using github because a lot of code that you'll probably want to reference is there for you To inspect and other people Especially in security if you're downloading a security scanning tool, you're probably going to go on git lab Or excuse me github to get it So you'll probably use github no matter what yeah, and it's one of the challenges of moving is in for example Even if I were to move over to git lab and move my dot files, they're not that they're that popular or not They're that special mostly a copy from cali linux. I even say that in my notes But one of the things about it is I now have all these links everywhere that are all set the poll from that If you switch where you have something and people don't switch where they pull from it unless you delete it They won't even get an error. They just think there's no more updates to that They just say oh it doesn't update anymore because there's nothing new to update So the pain of switching once again can be it's not that it's an insurmountable task But you somehow once you've been using something for a while you've built all these Scripts that implicitly linked to a specific url matter of fact one of the conveniences that You know you can list your public ssh keys over in github and once you've got a bunch of scripts that pull those ssh keys in You're like, uh, I have to get where did I put all those scripts? Because I just started setting them up everywhere to make sure my latest ssh keys are on servers Uh, I have to go find all those servers and make sure they're pulling from the new place I keep my ssh keys. So switching can be hard in general too. So migrations in general are a pain. Yeah, um, there's there's a third solution though I want to mention actually sure and it's called git t g i t e a Like the word git and t like i'm going to drink a spot of t in one word So git t with only one t And it's a self-hosted Solution for this that is really good. It's one of my favorites. I don't need again I don't know why I switched to git lab but git t was fine Maybe I just wanted to try the other one It might have been a work really the reason at the time because I think my company was getting into git lab That could have been the reason But git t is written in go you can install it on your own server So you can have it on your lan if you want So if you don't like the cloud you could just keep it local if you you could write it on a linode server Or whatever if you want it to be remotely available. That's up to you Just know if you make it make it remotely available security's on you right because you got to make sure it's secure And updated so you're taking that responsibility upon yourself But git t is a lot is really great you install it on whatever linux server you have local or not And you could just use that for where you push your code to And um, I used it for a while and I I had no complaints with it. It was great I might make a video for it if there's enough people that want me to do that It's an alternative to github and git lab that might be something that'll be right up someone's alley So it's uh, just git t.com gitea.com. You could check that out. Look at the deployment instructions How do you deploy that? Um, I don't remember the process, but I do remember it seeming like very pretty easy You're just downloading go and then the module and you're just running it essentially so It's pretty easy to do that sounds pretty simple Yeah, yeah, I thought it was too so I really like that one. Um So Where do where does this go right? So you so you have all this git thing you learn it Your your version controlling your servers or dot files Maybe you sort of get t or a git lab account or something and you have somewhere to put it I think that this organically You know most people unless you start with with um automation Maybe you're getting into home lab and you're like man public looks cool chef looks cool ansible looks cool And you learn it straight away. I think if you don't do that most people naturally kind of evolve into that from this So maybe you have some dot files that you put in git and then there you go But then later on you you start to look at your version your automation tools Let's just use ansible as an example something we covered before You keep your ansible code in there You can make it you can make the repository private even on git lab and github so no one else can see it And you could have like a deployment key that only you have to download it So you don't have to worry about like everyone downloading your config files or you just run your own server local land and um I feel like it once you put your ansible code in there It starts to kind of make sense how this would work in a company because a lot of home lab stuff It you know might if you work in it as well You can translate to that so then you could have like You kind of know what it's like to use version control So if you work at a company that's using puppet chef ansible or something You have multiple administrators that are pushing code to that repository so you could kind of get used to the workflow of peer programming essentially and I think that's a Pretty cool thing, but where it all comes together though is ansible pull And ansible pull is my personal favorite way to use ansible and just as a quick, you know refresher because we talked about it before In my case, I have laptops desktops and servers that are maintained by ansible My laptop could be in my bag I mean, which means it's it's asleep and it's not getting any network connectivity So if I have a ansible server and it tries to reach that laptop when it's suspended It's not going to it's going to send me an error email. Yes. I know I put the laptop away. I'm not using it Leave me alone But you know, of course the ansible server is just doing his thing is trying to reach a server and it can't but ansible pull is the reverse where The laptops the desktops the servers they pull down the configs from somewhere online So they're only going to pull it down when they're online or just powered on So that way, you don't worry about getting errors when it's off. It just next time it comes up It's just good. It's running via cron. It'll just download what it downloads if it's if you have that code in a version control Utility anyway or server. It's very easy for For you to basically migrate that ansible pull because essentially you're just changing your deployment script to pull from a url That url could be the repository for your get lab Server get hub get t whatever it is and then all of your servers can download from that central location So the way it started for me like, you know, I did seem sim links to dot files But eventually ansible just has these dot files and just make sure they exist on all my machines And it's all very organic the way that that kind of Played out And into current day to where I version control and automate everything I think it kind of it's a symbiotic relationship I think is the best way to put it when you're dealing with configuration and version control Well, and it's it kind of goes to the me and g we're talking last night and one of the challenges sometimes is when You don't use a laptop for a little while not just now because there's a change But maybe you didn't need it for a week and i'm looking over across my studio table here So you know what I have the same thing I have that laptop probably hasn't been on in two weeks because it's just a lab laptop that I use for some of the demos So having the ansible pull tied to all of the get versions because as I've made changes since then to different things I'm working on but when I need to fire up one of those demo things It's like pull it open it up. It's going to do the ansible pull It's going to grab everything it needs and put it all together and deploy for you That's the I'm not quite there because I'm not using it. I understand that more in concept j actually is using it So I'm I'm with all of you on this in the audience here learning some of this and getting better at some of the automation Because it's not my uh highest skill set right now Yeah, well, that's okay because it isn't your highest skill set until it is right? Yeah It's just it becomes I can become better and yeah, um But so so what I want to do is talk about ci cd continuous integration continuous deployment Someone corrected me and thank you by the way in the in the chat room Um, you know, like I said, I see i cd. I that's what I say, you know After a while he kind of lose track of what it what it is But um, I know what it is in theory because I use it Um, when we have the ci cd tools Just to kind of keep it really simple you you'd have something like jankins, uh team city, um I there's others Uh, I think i'm gonna there's probably another really common one I can't remember the name of but anyway the idea is it tests the code for you I'm making it very simple because it's much more than that But you commit code to it it can run the code compile it It could just you know The agents kind of do the thing with the code and verify that it is actually good And then you can use it as part of your deployment now That's not necessarily something That's going to really translate the homelab all that much But there is overlap that i'm going to mention which is vibrating enough so With ansible pull the issue became that if I commit a change Because any commit I I make to that repository is going to be run on every single machine So at some point I had like 20 30 of these that are checking it So if there's an error syntax error Then i'm going to get 20 or 30 emails from every single server complaining I it didn't work because I have this you know catches standard error and you know It's going to send an email to me that there's been a problem It gives me the error message and then I get like 20 30 emails and um, okay And I fixed it and I get 20 30 email saying that it's successful, which was just getting annoying So I I made kind of like a poor man's ci cd solution So most people will probably use jankins team study or one of the others Have a server they have an agent the agent runs the code But what I did instead is I I added another branch called staging to my ansible Repository, which is just where I test things and I have one candidate For for every single operating system. I run. So there's an arch Linux first, um, bm Debian ubuntu pop o s menjero All of those and they all pull from only the staging branch And so when I make changes I make my changes to that branch push it up there So only those servers get the change because I have a representative of every distribution So if it works on all of those I don't get any error messages. Okay, great I could commit it now to the main, um, branch and then all the other systems get it Now This isn't a true ci cd solution because there's no ci You know server here each one of them is just checking out a branch But they're reporting the success or failure to me so I can basically find out if it's working or not So that way I don't deploy something to like 20 or 30 machines and break them all And or worse lock myself out, which I've done It's a lot easier when you lock yourself out of like a small subset of staging servers Then everything and then you have to actually access a remote console You know Because you just put a empty variable in the ssh config file and now the ssh service won't start anymore which is Um, but you know, that's kind of where it goes because ci cd isn't one of those things that we normally think about in Home app, but if you have automation like ansible You might want a vm That's a representative of the distro or distros that you run that could just run a branch report success or failure So you have an idea. Okay. This is exactly what I think it is And then you could push it to main and everyone who gets it. So I think that's where um ci cd might have a possible role and Some of these tools like git lab and github they have ci cd built right in So you don't even have to do the poor man's version like I am you could literally just Create the agents right there inside that service and do it the proper way rather than the um, you know rigged way that I'm doing it So that's just a tip for some people out there Maybe only a certain subset of our audience might find that helpful But I think it's better to Break a few staging servers than it is to lock you get locked out of like your actual production servers Yeah, and a lot of this goes towards the same thinking that a lot of people that are building home labs Some of them are just hobbyists, but there are a lot of people that are going You know, this is my career and I need a place to sharpen my skills learn the skills And if you're going to go into a collaborative larger scale environment at a large company These are tools and skills you really need because in order to collaborate on this It's not just you making changes. It's not just one person It's a team and some larger scale teams really need a lot of this because you need to know who changed what when And will those changes Scale out and you know when I look at one of my friends manages over 150,000 end points At his job. So small things have large ripple effects and you know, yeah Yeah, and sometimes see it's the question of what went wrong because this seemed to work in staging and 150,000 end points. It's not working on 25,000 of them And what are those commonalities and this is you know, some of these bases you need to do if you want to get Into that type of position or those type of jobs someone in the comments mentioned that git lab has a CC ID that can be run on site So I guess they have their own tools for it and someone else says the azure has one as well Um, the azure one's slightly relevant and I won't get far into it, but look up the new, uh, omi It's called. Oh my god is the new hashtag going on twitter for, uh azure Probably not testing their builds properly and backdouring things into linux through their process of images and Based on based on how slow they're working. I don't think their dev ops tools are working very well Or at least the people that are in control them, uh, aren't deploying this very well So that was a breaking news story that happened just before we started the podcast But it still goes into the same thing because now people have to go look at, you know How do we get this out there? How do we push it? How do we get this out to all these different servers that it's in? And push the update for it So and that's where I think that's a good point because that's where this kind of comes together because if you have your version control Or you have your config management scripts and version control then whatever the fix happens to be You could add a playbook for that and push it push it into your repository and then all of your servers get the fix done That's what I do. So if there's ever a security vulnerability, I think I can't remember what it was. I think there was um There's something that happened and that's the sage I think over a year ago or so But whatever it was, it's like it required a specific package So I just made made it a point to it to require that every server was, you know, running at least that version Across the board and of course I had it set up to restart SSH as soon as it's updated So rather than logging into every single machine I just do one commit technically two because I tested on staging first and then It proved itself there merged domain and then everything got it. So um, that that was just super simple And I don't know how we survived with logging into everything one by one I can't even imagine what that and I've I've experienced this like it must have been so traumatic That I've like blocked it out of my mind and have no memory because I know I did do it that way But when automation took over like that was deleted and okay, this is the only way to do it for no I'm not logging into hundreds of servers anymore. Well, honestly some of the um ease of automation goes hand in hand With linux a lot smoother than it's gone with microsoft You know, even with some of the advanced tools we have pushing updates to microsoft servers is constantly a less than simple process Because it's so fundamentally built built different, but linux has been always on this side. So it's it's definitely um Once you get into some of these other skills like that a building things with ansible It does make it a lot easier to do that and I think it's easy to forget what it was like logging in With the exception of the things that still don't have full support for such things like special distributions and stuff like that That still require manual logins to push new versions and updates. So Some of those I mean with ansible, you know, there was a one at one point I can't remember which distribution it was In ansible that wasn't actually recognized and I literally pushed that in myself I wish I could remember which distribution it was so I could take credit for it Before the because of the fact that ansible supports that distro is my fault But literally it was one of the offshoots. So I was either a spin of a boon to a spin of art I don't remember but essentially it's just one file and you just say, you know, this distro matches this one So it knows this uses dnf or Um yum or pacman apt or whatever and I just pushed it right in there and I think that's the right mentality But um distribution support generally happens pretty quick in ansible, which is pretty cool Yeah, and someone says using it just to automate your switch and router config backups And getting rid of rancid makes it works Uh makes it worth it rancid is another tool for doing configuration management for those of you that don't know I've never used it. I'm just aware that it exists. So yeah, just like me and stowe Like a lot of people seem to like that one. So um Again, I mean to bring it all back. I think version control is essential for everyone that's doing anything linux related Even if it's just your notes That you're keeping track of or you know your server configs and just keep in mind if there's any situation where You're like man, I put a lot of work into this Uh configuration for this server it would really suck if I lost this config and had to start over Get that inversion control absolutely put that inversion control Again, make sure there's nothing there that shouldn't be inversion control ignore the files with keys and API credentials passwords and hashes get all that out of there And then just upload the rest you could you could you could reset your password? It's not the end of the world um and your api keys But the rest should go up there and especially I think you'll be justified that one day where your server's broken It doesn't work and then you do a get status. Oh, I see what changed And that saves you hours of just going through every line because you don't know which line changed Get will tell you which line changed and you'll know exactly what happened at that point And I think at that point is when you're really going to appreciate it. Yep All right, I think that's You can get started with uh, I mean there's a million different get started books I don't know if I have any one pick or recommend But there's there's lots of resources out there if you google how to get started We get and starting your first couple hello world version of your first get there's plenty of information out there I don't have any specific resources to link to but um, let us know down in the comments If you're watching this on youtube of some other things on get it maybe uh, you want I'm gonna point at j maybe j will cover in some other videos because it's it is on my list It is actually on my list. It's not I don't know when I just know it's not going to be this month or october But it is something that Um, I might try to push it up if there's enough demand for it But it's definitely going to happen So I will have a video and I might do a video on get to if there's enough people that want me to uh to do that Yeah, because jean's been busy if you haven't noticed he's got some uh whole new Series of videos on proxmox out there a lot of people ask about it We did a proxmox video but people said but tom proxmox seven is out so j did some Deep dives and put together a series of tutorials to get your proxmox fill and Wendell on the other side of it because he was a guest on here. He's starting to play with xcp and g. So I'm excited Awesome, that's really cool. Um Yeah, maybe somebody else start playing with that again I did enjoy it when I looked at it. So um, it's a good solution So respectively on each of our channels including level one techs Um, we're all we're all cranking out lots of other stuff out there But let us know what you want to know about it and uh, thank you for joining us today Much appreciate and ask us questions so we could do another q&a at some point. That'd be pretty cool We do enjoy the q&a. So we'll lead up to another episode. There's a Form you can fill out on the homelab.show. So you if you have your questions you want to ask there We have forums Ask away. We do look at the youtube comments as well So wherever if you're listening to this you can do it Do the website if you want to come to the youtube channel you can do it there And me and j both have forums directly. So we will consolidate stuff and now we'll do another q&a episode in a few Yep, looking forward to it. All right. Thanks everyone to join us. Appreciate it. Thank you