 Okay, cool, let's get started. So today we're gonna talk about several smaller topics. So I'm going to talk about .files, then, because I was gonna talk about backups and the OS automation. Then John's gonna talk about machine introspection, like OS monitoring, kind of single machine that hops. And then I'll talk about program introspection, just like debuggers and profilers and that sort of thing. So for our first topic, we're gonna talk about .files. Which is a topic we actually touched on briefly in the last class on editors, when we looked at our VMRC files for configuring this editor. So a lot of these command line programs are pretty configurable using plain text files called .files. The reason we call .files is because their file names literally begin with a ., like this VMRC file is the file of period VMRC in the home directory. And the reason it's like that, there's a kind of interesting history behind it. So if you notice, like if I go into my home directory where I am right now and I type in ls, I don't see that file, even though it does exist, like if I do head to the slash .vmrc, like there's some content there. So why don't I see it in the directory listing? Well, the file is hidden by default. If I do ls-a, a is for all, it shows me all the files in this directory, including all the things that start with a .. And so this is a kind of useful thing because you put stuff that you don't wanna see in directory listings all the time in files that start with a . And then it'll bother you, but you can find you if you need to. The reason this is implemented like this is almost like a historical accident. You can read more about it if you're interested. But what happened was that early versions of the ls tool had a thing where they hid the directory entry for the current directory. So if you do a directory listing with dash a, you'll see that every directory has . and .. And what those represent is that . is the current directory and . is the parent directory. And when we normally look at directory listings, like we know that there is a directory that we're in. Just having a . there doesn't really help us. But if we look at, if I do dash l, it gives me some additional information. If I wanna get this additional information about this directory dot, well, it can be kind of useful to have there. And so there was this line of code in the C code for the ls tool where it compared the first character of each directory that it was about to print or each file that it was about to print. And if the first character was a ., it skipped it unless you had the dash a option. And so this had the side effect that any file that had a file in that happened to start with ., even though it wasn't . or . . the current directory or parent directory, it was still skipped. And so then people started abusing this feature or this kind of like implementation bugging ls in order to use it to implement hidden files. So that's why .files are called .files. And so all these different command line tools have configuration files that often live in the home directory, things like that .vmrc or files like .gitconfig. So we also looked at the git version control system last lecture, John talked about that. That tool is configurable through this file that lives in the home directory called gitconfig. And you can pretty heavily customize if I'm writing a lot of stuff in there. So different tools are configured in different ways. So to learn about how to configure any particular tool, you'll have to read that particular tool's documentation. Sometimes it's through the programming language for the tool. So if you wanna configure your shell, well, your shell just reads a bunch of files on your disk and interprets them as shell scripts on startup. Similarly like for .vm, .vm is programmable through a language called .vmscript. And so the .vm configuration file is just a .vmscript file that's loaded on startup. So sometimes these are full programming languages that you can use to set the different settings for your program. Other times like for files like the gitconfiguration file, this isn't exactly a program, it's just a particular file format, and you just set a bunch of settings to different values. And so we think that customizing and adapting your tools to behave exactly the way you want to is totally worth investing time in. And one thing that some people do is they go online and find configurations that other people use for these tools and just kind of download them and use them as it is. And we recommend against doing that because you really should figure out like what you want your tool to do and customize it so it best fits your needs because your needs may not be the same as somebody else's. But it can be helpful to figure out how other people have customized their tools just to see like what kinds of options are available. And so you probably have some .files set up already. Like some places you could look. Like you can go on your local machine and look at the file .bashrc in your home folder. Does everybody understand like what these Unix style have passed me? Hopefully people at least this level will be exposure to Unix. Great. So you could look at this file, you probably have something there. Maybe some minimal configuration where you say setting your bash front to something. You looked at these in the shell lecture. And now if I run bash, it will interpret that bashrc that I wrote and set my prompt to this thing that I just changed it to. Other places to look are the files like tilde slash dot emacs for your configuration for emacs or tilde slash dot vim for setting for vim and so on. So yeah, the different ways you can learn to customize your tools are you can go online and just look up to tutorials that people have written. So oftentimes these tools are really heavily customizable. You can look at the manual pages for them to see all the different options. Like you do man and then the name of the tool. Sometimes it'll list out all the different configuration options that can go in the configuration file. Like there's one particular programming that's called axl which downloads files from the internet. And it says oh like tilde slash dot axl or c is the file that we use for configuration. And for this particular tool it says that the settings that bring are not documented in the man page but it tells you where to find those settings. But yeah, other things you can do is you can just go in the internet and Google like axl or c and then you'll find different people have configured their programs to be in certain ways and oftentimes have written some of the documentation for what these different configuration options do. Another thing you can do is just directly go and scan through graphs through people's dot files. So like if you go on github you can go to the search bar type in dot files and a bunch of people have repositories on github containing all their configurations and then you can just go look at them and be like oh let's see what this guy did. Looks like he had a pretty popular repository. And then you can go and see like oh what are different ways you can configure git. Well it's in the git config file. This is all documented. You can kind of pull in one of these configurations at a time and just see what it does. So it's a good way of exploring things. Okay so that's the very basics like what is a dot file and why you should bother to figure out how to configure your tools. But now we want to talk a little bit more about how you should organize these files and kind of different things you can do to keep your settings consistent across different machines you might use. If you put a lot of effort into customizing your setup on your local machine, say your laptop and then you're going to work on a different desktop computer somewhere or you're working on a theme or something you don't want all your software suddenly behave really differently, right? Once you get used to your super customized setup say customized key bonding for your editors and lots of nice shortcuts for your shell and so on. You want that configuration to be kind of easy to replicate across different machines and easy to keep them same. And so yeah, next we're going to talk a little bit about how you should organize your dot files. But before we do that are there any questions about the basics like what is a dot file or why you should bother to do it? So there are like programs like Souveness that have ST so they don't have like dot files so how do they like work? I'm not familiar with that particular program. Can you repeat it? So like there are some things that like they don't have dot files like ST so they're like Souveness or anything? Yeah, so the Souveness tools usually the configuration is compiled into the program. Oh, okay. So you actually, the way you configure it is by editing the source code of the terminal and then changing the settings you want. It's extremely painful but it doesn't mean that you have a, it doesn't mean that you can change whatever you want. So the idea of the Souveness tools is basically that they're all just a single file. So they should be relatively easy for you to maintain but in terms of configuration they're pretty painful. Another related issue like for these command line tools they're all configurable really nicely through these plain text files but for other programs you might use on your system like say your web browser or something doesn't really use a plain text file for configurations. It's a little bit harder to replicate configurations across machines but they're more specific to what you can use for that. So for your web browser and sign in to my Google account and use that to synchronize my configuration and things like that. Was there another question? Although that's not, there are like some, like you can find like some tools like for example like I'm thinking of NDB like to play videos that do use like dot files instead of just I'm like a settings window when you're still like picking text boxes. So like really convenient to just kind of compile and like use them. Well, okay so next let's talk about how we can organize our dot files. So hopefully by now you're convinced that it's like it's worth customizing your tools but now yeah you might use multiple machines or even if you're using a single machine it's really nice to keep track of all your settings in some principle way. So what are some goals you might have in organizing your dot files? Well, one thing we should do is we should make it really easy to install. So like if I go into a new machine, it's the one that I've never used before all of a sudden all the tools work differently than I'm used to because I've customized my stuff very heavily. Well it should be the case then that I can very easily install my own personalization on that machine. And so maybe a goal we might have is like we keep a bunch of stuff under version control and then to install it we should just be able to clone a repository and run a single script that everything set up the way we want. So easy installation is one goal. Another goal we should have is portability. So you want your configuration to be the same everywhere it should be easy to synchronize back and forth between your configurations on different machines. So if I use my local machine and then I say accessation to it via network for ease of use I'll pass it down to one of my own machines. Like everything behaves the same way on this machine as it does on my local machine. And if I make a change on my local machine it's easier to synchronize over to this remote machine. And then the final thing is I think it's really important that you track your changes in your configuration. If you think about it your configuration of your machine in a way is like the longest project you'll ever work on, right? Like for your entire programming career to be using these different tools and you don't probably be filling with the configuration as you go. And it's just really nice to have all that under version control so you can see what you changed at a certain point or why you changed it or things like that. If I go to my own .files repository for example and look at any particular file in there I can go and figure out, okay when did I add this particular alias? Forget. I can go and say I go back in 2013 I made this particular change where I added a bunch of stuff and here's why I changed it. It's just really nice to have that history. Sometimes you make a configuration change and you realize you don't like it you want to go back to an old version. Having all this history there is super useful. And so how do we achieve these different things? Well so I think one nice way to organize things is have a single folder that contains all your configuration have that folder under version control. Maybe put it on GitHub as a public or private repository depending on how you feel about what you put in there. And then have a single script setup so that when you run that script is like type in the name is script press enter and it configures everything. And then if you make that installer script item quote which means like if you run it multiple times it doesn't do anything bad. Well then synchronizing your .files across machines is really easy. If I make a change on my local machine I add the change and then push it into the git remote and then on my other machine I go into my .files folder do a git pull and just run the installer script and it will set everything up for me. So just to show you a little more details of how we can go about setting something like this up I can just demonstrate it in front of you. And this is all in the lecture notes we don't need to remember the exact commands I type in but they'll show you how you might wanna organize things. So first I'm going to go ahead and make a directory called .files where I'm gonna keep all my settings. I'm putting it in a location where I'm gonna get rid of it later because I already have all the stuff set up but here we go. And then okay so all my settings are gonna go in here and I want to keep this entire directory under version control. So we talked a little bit about git last lecture and that's our tool of choice. So we do git init and that was a setup as a git repository with no commits yet. And now I can start migrating over my configurations I've already set up on my machine but now for now let's pretend that I have no settings so I'll just start creating some settings. Say I wanna configure my shell so I'll create the bash rc file and I can start putting my settings in there like say I want to customize my prompt so remember from our shell lecture bash looks at the ps1 variable and that kind of tells it how to draw the prompt and so I might want it to show the current directory I'm in and then like a dollar sign and then a space and that'll be my prompt. If I open up bash and type in this thing that's my shell prompt name. So okay now I have this directory and it has a file in it called bash rc but this is not where bash reads its configuration from. Bash reads its configuration from the file till the slash dot bash rc. So how do I reconcile that? Like one thing I could do is instead of keeping this dot files directory under version control maybe I could just go to my home folder and then set up my entire home folder as a repository and then track the specific files in there. Turns out that's actually a bad idea is anybody know why? So if I want to be tracking the specific files like just my program configuration like dot files if I keep my entire home directory under version control well there's a bunch of other stuff in there, right? Like my photos are in there and my other Git repositories are within there for all my programs and stuff and so I can set up like this top level Git repository to like ignore all the other stuff underneath it but it's kind of easy to use the wrong Git command and accidentally delete stuff inside a Git repository that you're ignoring. So it turns out that's not a super great approach. So instead of what we think is a better way to do things is to in a sense like have a folder where you have all your configurations and then copy all these files into the right place. So like if I copy this bash rc into tilda slash dot bash rc and then like now we're on bash I see my shell prompt that I've configured in this bash rc file because it's reading it from this copy. But there's a problem when you use copies what's kind of annoying about it is that well now you have two copies of the same thing and now you have to worry about keeping them in sync, right? Like if you just directly go and make changes to this version well it's not going to update this version which is the thing that's actually inside your Git repository and is you're tracking what different versions of, right? So there's actually a nice solution to this. It's something called symbolic links. And basically what you do is like your file system has support for kind of thing where you have a file that's just a pointer to another file. It's like a shortcut for alias. So I'm going to delete my copy of bash rc and instead use the lm command. This says create a symbolic link. And I'm going to create a symbolic link to this like actual data that's in my Git repository and give it the name tilde slash dot bash rc. And so if I look at a directory listing for my home directory I'll see that bash rc now points to the actual content that's tracked in my dot file repository, right? And so now I'll see if I load up bash, oops, not bash rc, the program's called bash. If I load up bash, it does read from this configuration file even though the file is not actually here because there's a symbolic link. So when bash tries to open this file it just kind of transparently ends up reading from here instead. Okay, so now we kind of know like we have a single folder where all our settings are and then these settings files need to end up in different places and we create these symbolic links to make pointers in the right places to point to this underlying content that's in our dot files directory. But having to do this on every machine is kind of painful, right? Like we might have a bunch of different configuration in here. I might have a file vim rc for my vim settings and I might have a file axl rc for my axl settings and so on. And if I say copy this dot files folder I'll make settings to a different machine and then I want to apply all these settings while that involves copying or meaning symbolic links to these files one by one and that's kind of annoying. And so what we should do is we should create an install script that does this linking for us. And so that's actually really easy to do. I'll make a file called install and make it executable. Remember, we saw a little bit of this during the shell lecture and I can start editing this install script. And it's actually pretty simple. So I'll explain what this does in a second. Okay, so this is all the script needs to do. It needs to have lines like this. So first it figures out what directory the script is in and then changes the current directory to that directory. And then one by one the script just calls the line command to set up all the symbolic links. So what this line says is okay, like look at the current directory and take the bash rc file that's in the current directory and set up a symbolic link in tilde slash dot bash rc. And then I'll do something similar for all my other files. So I have a file in here called vim rc and I want to simply get to the file tilde slash dot vim rc and so on. And so now once I have the script, setup is really easy. I can just run the scripts and it will create the symbolic links for me. What's kind of annoying about this particular setup is that you can try to run the script again and again. Well, if the file's already existed, it complains and so now we can go back and write a slightly more sophisticated script. For now, we're just going to ignore the errors and it will, you can also redirect. Yep, put up this thing. So it throws away whatever error message is printing and now we have something that kind of looks like it works. Now, this is actually the ideal approach to use and there's some slightly more sophisticated tools you can use instead of writing your own script to do this whole linking business and we've linked to those in the lecture notes. But at a high level that's how these things work. They just take files that you give them and simulink them into the right place. And now remember, I've set up this whole thing as a gift repository so I can start actually committing to this repository and now I have history of my changes. And so I made my first change to my .files and I can go and fiddle with my setting some more. Say I want to change my batch prompt to say something slightly different. Maybe I don't like the dollar signer. I want to change that to this angle bracket and go make that change. If I go here to get status, I'll see, okay, I've got modified some file and I can go and say update batch. And this way I have a history of all my changes. If I now upload these on GitHub, I can set GitHub up as a git remote and then just do git push to synchronize all my changes on my local machine to GitHub. And then if I'm on a different machine, I can just do git pull and then the URL for my .files. And then to set it up, I can just go into the directory and run the install script. And this is kind of like to set up a new machine, I just do a git clone, cd into the directory, .flush install and then it's done. And so this is how you can have your configuration replicated across machines and nicely synchronized and track out the version history. And so everything we've showed you here with the exception of this install script is I think the right way to do it. Instead of writing an hacky install script, there's some slightly nicer programs that people have written. And we've given you links to those. And so it's totally worth setting this up in the right way. I'm sure it will be your longest-lived software project you've ever worked on. Yeah. I just wanted to share that Anish is being a really humble, you know, creative dot bot which does exactly this. It's on GitHub. He's like one of the creators of one of these things. Yeah, so there are lots of tools that sim link your files with the right place. I wrote one of these because I didn't like what was already out there, but there are lots of different tools that have slightly different ways you use them. We've also linked to me, John, and Jose all have lots of configuration files we've written and ours are pretty recently well documented. So we've linked to these in the course notes if you're interested in seeing how this setup has been set up with the shell prompt and having the git status and sign up whatnot that's all available for you to still run. Do you have a recommend way for testing? Like, for example, let's say you have a virtual new Mac. It's not all of us can just like, grab a new Mac and test our setup on some new Mac, right? I mean, this is being out by VM. I mean, just like, go on this grid and see if everything works. Yeah, do you have a recommendation for it? I think you can install macOS in your VM now, right? Yeah, but also like, okay, so I think I don't really bother to do that. I'm just, and I do play around my configuration sometimes. I'm just somewhat careful with what I test out to like, try not to break things so much that I have to reinstall my OS. I think for most things, like, you can probably just do them on your current setup and it's fine. And especially when things are under version control, like, pretty easily, play with stuff and then revert changes if you don't want them anymore. Yeah, good questions. Any other questions? Yeah, and if you go look at people's projects, so we've linked to these, you can find lots of .files online. You'll find like, there are tons of configuration files from many different tools and every tool is really heavily customizable. Well, for a lot of people, they've been working on this product for many, many years. It's kind of cool to look through. Different people will have different ways they have a one-click way of installing their .files. So here's a kind of analog to the install script that we were at except this one actually does a lot of things better than ours. Like, we'll be careful about overriding files and things like that. So it's totally worth taking a look at these things and setting them up properly. I'll briefly talk about one more topic that's just kind of worth knowing about. One thing you might want to do once you start using multiple machines is have slightly different configurations for a machine. Like, for example, I use kind of overall the same configuration on here, but also on Athena and also on some machines I have in my room. Some have different hardware, different settings for that hardware. And so my .files are like 99% the same between all these machines but there's some tiny changes. And so how do you deal with that? Where you like, want to have a good repository and kind of like nicely tracks changes. And so we've written a bunch of details on this in the notes. You can find those online but I'll just briefly mention some of them now. Some things you can do are within these configuration files, for a lot of them, they're not just like configuration just data but it's actually programs you're writing. Like for your BatchRC file, it's just a batch script. And so you can have conditionals in here. It's like one thing you could do is like here I'm not writing actual code anymore. I can do something like, oh like if this code is running on like my laptop then do something else if post name equals like GPU machine do something else and so on. So you can set up conditionals in your settings to do different things based on the environment. Some other approaches you can use a lot of these tools support well like reading and executing code from yet another file. So one thing you can do or like one thing I've done for example is in my GIF config, I have all these settings that are common between machines. And then at the bottom of my configuration I've written this include command. And so what this does is it says like, okay look at this other file and process the content from there too. And so I can have this file be common between all my machines and I can have small machine specific tweaks if I need them in this particular file. So those are two approaches you can use to deal with different configurations on different files. Yeah so lots more links to resources in the course notes and of course feel free to post on Pianza about this if you have any questions. I'm also happy to take any questions now if you have one. Cool. Okay so next Jose is gonna talk about some things and then we will have a break.