 Okay, did anybody have any questions or anything from last time? Or comments or anything else? How's your VIM going? I just had one, Jeremy. I wasn't really sure what happened when I looked. So I did the VIM and CTags thing in paper space and as per the session. And then when I shut down the machine or the instance and went back into the same machine, it didn't seem to work. So I'm not sure why or if you want to go into that. Okay, let's have a look. See if we can replicate it. Yeah, what might have happened? What happened when you tried to use it? Sorry, I muted myself. So what happened was the member and I had to reinstall member. And what else was this and CTags again. And the CTags file had changed. Actually didn't have anything in it. Compared to how it was when it was originally created. So I don't know what happened. But all right. And then when I tried to reindex or recreate index, and that didn't seem to work either. Well, I mean, one obvious thing we should probably try to get going is being able to see the same, you know, binaries and stuff being installed. Having to reinstall CTags every time is certainly not ideal. So let's try and figure out how to do that. I'll share my screen. All right, can you see that okay? Let's start this one. This is the one we created the other day. So the only things that are going to still be there next time we come back are things that are in slash storage. Or things that are in slash notebooks on the exact machine that we just started. And to remind you there are, you know, we can make this more flexible by taking advantage of the, what's it called, pre-run.sh and the .basrc.local scripts that we can put in slash storage. And we can use those to copy, link or move stuff out of those persistent places into our home directory if we want to. So that's kind of how we've been organising things. And so when we do pip install with a dash dash local flag, it installs things into our home directory into a subtotal.local. And so that we have moved into slash storage and then semalinked it back again and now script back to home. And so now anything that we install with pip install minus minus user will be persistent across all of our sessions. Can I ask a question about that? So does that mean that if you give in the past suggested that you can wipe your Mamba directory and use it to reinstall? Will the fact that you have the semalink to local on the storage drive, will that mean that those pip installed files will be persistent even if you wipe your Mamba drive? Yes, although that thing about wiping was not for paper space because remember on paper space the Mamba stuff is installed into the root directory into the slash opt or whatever it's not in our home to like usual. So you can't really wipe Conda on paper space. On paper space, Conda is always wiped clean because you start with a new server. And so the only stuff left will be stuff in.local. But yeah, I mean I don't know why you would do the same thing on your own machine. You could pip install minus minus user and have a local directory that would be separate from your Mamba environments and stuff. Okay, I guess it's moved. Yeah, I hadn't thought of it that way. Okay, thanks. No, that's fine. So I'm just having a look. So I think the way we want to install stuff like CTags, which is not a Python thing, is using Conda. And so we can install with Mamba as we've done before. Now, by default, that's going to install it into the slash opt slash etc, which we will lose. But what we can do is Mamba comes with a flag we can use which is minus P for prefix. And if we use the minus P flag, it will install stuff into a different directory. And so we should be able to use that to install stuff in a way that we can that we can reuse it. And I think this is going to be a good exercise in understanding paths and aliases and similar links, I should say, and all that kind of thing. So I think, yeah, I think this will be an exercise worth doing. So I'm just loading up Gepidolab. Here we are. And so this is the same server, notebook server that I used last time. And since slash notebooks is persistent, that fast folder that I similar, that I didn't similar, that I get cloned is still there. So if we get a terminal up and we can probably take advantage of our space and then Apple B or control P B shows and hides that sidebar, which is quite useful. So what we could do is we could create a, so I just type CD which is moving me into my home directory. And so we could create something to put other condo stuff in here, we'll call it condo. And I believe we could go member and then say a prefix. So a prefix is going to be our home directory, condo directory that we just created install, universal dash ctext. So, okay, so all right. So we also don't have member installed, which is a good point. So we'd like that to also be permanently available. So let's start by saying condo install minus P condo minus P install member. Okay, that's probably going to be after the install command is my guests or minus P member ups, minus P tilde slash condo. Okay, so well that's going on. Okay, and then because they're not using member forge, that means that the condo forge channel is not available by default either. So if you use member forge, then it kind of assumes that you're always going to say, you're always saying minus C condo forge but since we're going to, and I think the plan is a paper space is going to hopefully switch to using member forge soon, but for now we'll have to say minus C condo forge. All right. So while that's going on let's also get our fast book here which actually makes me think about something I just tried to reuse a command that should have been in my history and it wasn't there and I think the reason it wasn't there is that we, oh here we go, let's say yes over here but why is this installing Python even? noise. Oh, okay, that's annoying. So member okay, it's say, okay, so it's trying to install it into the tilde slash condo directory and it's saying, oh that directory doesn't even have Python in it so I want to install Python and everything into there. That's annoying. Let's say no. Okay, I do have another plan up my sleeve which is there is a version of member that is fully self-contained everything statically compiled into it called MicroMember which so we're just waiting a bit here, once we get this working we will make this all easier. So if instead of using member we say slash storage, so we can put this elsewhere else in a moment. So let's see if that would work. Okay, that's also not working. This is quite tricky isn't it? So maybe we need to figure out what a minimum environment is because we don't want to use up lots of our storage directory with junk that we don't really need. So what I'm going to do is I'm going to install all this junk and then I'm going to see what that actually installs and then we'll try to get rid of as much of that junk as possible. That's my plan. I'm not sure if anyone is trying to solve this as you are Jeremy, but I tried with the condo installed minus the tilde condo minus the condo forge mamba and I'm getting an error environment location not found, not the condo environment Yeah, that's definitely what I was getting as well. So okay, no channel specified. I think I might be getting closer though on this condo forge. Okay, that seems to be doing something. So we might be on the way to finding a solution. Okay, let's see what this condo directory Wow, it's got a lot of stuff in it. So I want to find I would I don't want to like have stuff that takes up storage base and I don't need. So the first thing to find out is how much storage base does things take up. So D use stands for disk usage. It shows you how much storage space things up. Minus S just gives you a summary per directory and H is as usual human eyes. So like show me megabytes and stuff like that. So it's easy to read. So this is going to show me the size of every directory and if I want it for the whole directory. Okay, so I've got 276 meg, which isn't terrible. But let's see. Minus 38 meg Lib is 203 meg. So it's mainly in Lib, right? So what I could do then is go Lib slash to see which ones. Now that's quite hard to read. So when things are kind of hard to read like that, I want to find everything that's like over a megabyte or more. So in the terminal, one of the really nice things you can do is you can use the vertical bar called pipe. And what pipe does is it takes the output of the previous command and it sends it to another command. And so the thing that you're most likely to pipe to is grep and grep searches and only prints things that match a pattern. I just want to find those things that are a megabyte or more. So there's a capital M for megabyte and then there's going to be a space after it. So just search for an M followed by a space. That didn't work. Maybe that's not a space and it's actually a tab. Yes. Okay. So here's all the stuff that's over a megabyte in Lib. And we could do the same thing for the directories inside Conda. There they are. So of that 203 meg, it's mainly Python, which I'm sure we're not going to need. Lib Python, we're not going to need. So we could rm minus rf Python. Let's see what else we've got here. So we've now got 161 meg, still quite a bit, still mainly in Lib. So this giant thing here, LibICU data, I don't know if we need that or not. So what we could do is just move it out of the way. So .70.1 and maybe just move it out of the way. And then let's see if Mamba works. Oh, we haven't installed it yet. So let's see if we can. So Mamba, okay. So we've installed C tags and Mamba, but I can't run them. Oh, that's interesting. I can run C tags. Why is that? Oh, okay. So it looks like paper space comes with an old copy of C tags. So this is the issue that you were having with not being able to use it. So normally dash dash version should tell you the version of things. There we go. C tags from Emacs. Wow. Okay. So my guess is it's not going to work the same way as what we're used to. And that's probably why some people were having problems. So this is not finding the one we just installed and we're not finding Mamba at all. And so to remind you, the reason for this is that the way that we the computer finds things to run is it searches in our path. And the stuff that we installed is in the tilde slash condo slash bin directory is Mamba. And here's our newer version of C tags. So we could run things manually by typing bin slash C tags minus minus version. Okay. So it does need that. Let's move that back in. I see you data. So that needs to go into web. Okay. And yeah. So here's a current version 5.9 compiled. Wow. Two days ago. So to make it so that we'll be able to see stuff that's in our condo directory we need to ensure that it's in the path. So to make sure that things are in the path when we run a bash terminal we have to put them into our .vim.bashrc.local file. So that's going to be okay. So it looks like I don't currently have one. So let's create one .bashrc.local. All right. So this is a file that needs to be run with bash. And what we're going to do is we're going to export sets of variable. And so the variable we need to set is path. And we want to set it equal to the condo bin directory in our home directory. And then we want to also have the contents of the existing path variable. So you have to put $path to say the contents of the variable. Okay. So now if we create a new terminal we can test whether that worked by printing the contents of $path. Echo is how you print things. Okay. Let's see. So that did not work. Let's try running it manually and see if it works. So source is a way of running a script and storing all the variables in this shell. So if we say slash storage slash .bashrc.local. There it is root condo bin. Okay. So why was it not working? Oh, it's not bashrc.local. It's .bash.local. That's why. So we could move. So I don't have to type .bashrc.local. I can just type exclamation mark dollar because that means the last token from the last line which in this case is .bashrc.local and move it to .bash.local. Let's try again. There it is. Okay. So now if I run ctex I've got the right version and I should be able to run mamba. Okay. So mamba doesn't like being run. So that's fine. What we can do is we can instead use this micro-mamba thing instead. So let's move micro-mamba into condo bin. So you can get micro-mamba micro-mamba. You can download it. All right. So let's see how we're doing. But let's get rid of mamba and let's see how much space we're now taking up. That's not bad, right? 175 megs. So we've now got a, you know, place that we can install software like ctex into and it will work. So to make sure that that will work next time we now need to move that into our storage and then of course we'll need to simlink it back again. So let's edit our prerun.sh file. Okay. We definitely don't have to remove any things. It won't be there. So we just simlink storage slash condo back to the home directory. Okay. So there's one other thing I'd like which is my .bash history file. It's really nice to come in each time and have the same bash history file on a machine in my opinion. So let's move that also into slash storage because that means that our control R and our app arrow and stuff like that are going to always work. So that goes into slash storage and so let's do that as well. So how am I going to do this in VM? I want to copy two lines and paste them. So to copy two lines to copy a line is shift Y. So to copy two lines I press to shift Y to copy two lines and then shift P to paste those two lines above. Okay. And then I can press shift W to move a word forwards and shift C to change the rest of the line. So that deletes and then it puts it in insert mode and I can type bash underscore history. This story. Now here's where it gets fun is I want to do exactly the same thing for this line is to replace this the rest of this line with .bash history. So to repeat myself I just press dot. Isn't that fun? So all right. Let's and that's not a directory so it doesn't need to be recursive. Okay. So let's try this out shall we by spinning up a new machine. I guess what I mean is spin up an existing machine with a new instance of it. So the reason we're spending quite a bit of time to kind of set up our paper space environment is like because I think it's like practical examples of using BAM and using links and using scripts, you know which hopefully you'll be to reuse these ideas lots of times. So let's start that one. Okay. So I'm just going to start up another machine and see if that all works. Okay. While we wait for it. So yeah, I have a feeling now once you've installed Universal C tags properly and it's in your path it's just a case of checking the version so let's create a new shell because that one's and I hit up arrow Oh, this is still not giving me the right version that's interesting. Not dollar shell to the path root condor bin that's interesting condor in C tags. Oh, okay. It's not similar back again because we didn't run that storage script because this is not a new machine. So for now, I'm just going to run the storage script manually. Okay. And okay, still not right. Oh, then the thing that puts in the path is meant to be the dot bash dot local, right, which should have run just create a new shell just to see if something weird happened. Okay. So all right, that's in root condor bin. So now if I type C tags minus, minus version with control R to find it quickly. Okay, it is working. So it's just a case of getting that running it again. Okay, so here's our new machine, which is now finished starting. So let's see if the same thing works on a brand new machine. Why is this take a long time to start open up to put a lab the first time for some reason on each machine. Here it comes. Okay, so fingers crossed C tags minus, minus version. There it is. And in theory, yeah, we've also got our history. So our history is actually saved between machines, which is quite nice, right? I love it. So if I wanted to echo path again, I could type control R capital PA and it'll find it from a history even though we're on a totally separate machine. One thing weird is we're getting the path here is repeating itself. Not quite sure why that's happening. Doesn't really matter too much to be honest, but I'm just curious about why slash opt condor condor bin is appearing. Oh, that's not appearing twice. Oh, they're all unique. Don't mind me. Okay. I was imagining things. So on this computer, I don't really want all this stuff in my notebooks either. So let me RM minus RF. Everything. That's a really dangerous command. Oh, Hamels here. Hi, Hamel. Yeah, I'm so sorry for being late. I forgot. I didn't even tell people you're coming so they don't know you are late so you could have stayed silent and not known. So Hamel hopefully is going to be joining most or all of our sessions from now. So do you want to quickly introduce yourself, Hamel? Yeah, yeah, I'm Hamel. I have, you know, work on fast AI a lot. I especially like contributing to all the dev tools, like NB dev and fast core and stuff like that. And, and like to other tools that help people automate what people do. So yeah, Hamels got background as a machine learning researcher and developer and also quite a bit of stuff with ML Ops and also training as a lawyer. So, you know, if anybody needs help with the law, he's a guy. Please don't. That's for the people about that Jeremy. All right. Nice to see you. So we're just setting up our paper space environment Hamel. And we've got to a point now where I just launched a new instance and everything is exactly how I want it straight away, which is really nice. We've got a way to install persistently Python modules. We can install persistently binaries. So I think that's a good place to start working through the book because we've got ourselves an environment. So that is step one. Is there anything else that people felt like we're kind of missing from the environment that they would really want? Or should we start working on chapter one of the book? Sounds like we're all happy. Okay, great. I'll stop this machine because that one's costing me money. So here we are in slash notebook. So the first thing we're going to need is we're going to need a copy of the book. So fast book is here. So we can copy the SSH GitHub thingy and clone it into slash notebooks. Actually, wait, we're trying to do this properly, right? So if we're going to do it properly, what we should actually do is create our own fork of it so that we can make changes and save it back again. So actually rather, so what I'm going to do is I'm going to click fork and that's going to create my own copy in JPH OO. Okay, so let's get rid of that copy in JPH OO. So we can actually do this all from scratch. Okay, so try following along with these steps. Make sure it works for you. So I'm just going to delete my copy of this JPH OO slash fast book so that we can start from scratch. All right, it looks like I need to go get my phone. Excuse me. That's annoying. Normally it lets me use GitHub on my phone. I'm just going to do this on my other computer rather than set this all up. This device option seemed like it was promising to me. This device? No, because like this is my new Mac that like I don't even know. Normally on my Windows machine I use Windows Hello to do face recognition. And when I say do it on my phone it's not doing it with the GitHub app on my phone which is what normally happens but instead it's not saying on my phone I have to install a USB security key so I don't know. That's right. I can just do it over on my other computer and delete this repository about that another time. Okay, on my other computer I should be able to use my security key. Yes, because it uses face recognition. Okay, great. I love Windows Hello. It's very handy. Okay, so now that's all done. So right now great. So now I can create my fork. There we go. So this is going to create basically a copy of the repository. But it's a copy that is linked back to the fast AI's version of the repository. So as if fast AI makes changes then I can click fetch upstream and it'll copy those changes into our version which is nice but it'll also keep our changes as well as long as they don't conflict. So now rather than cloning fast AI's version we will clone our version and we can save those changes back as we need to. Okay, so I'm going to go get clone our fork. You can see here it's got our username and then the repo name. Notice that I'm inside slash notebooks when I do that which means that now we can see it. And here it is. So let's open chapter one and let's open chapter one and clean checking. I remember previously with get clone you've done depth of one to not clone the whole repo. Yeah. I wondered about doing that this time but because we want to probably save like commit things back to our fork of the repo I decided not to go with depth one this time because we're not just reading it but we're actually changing it. I have a question. I have never done a fork and a merge. And I wonder if you can do a sample just a little test if that's okay. Yeah, let's make a change and copy it back. So let's just all the way to the master. Yes, yes, yes. Okay. So let's see if this first cell runs first of all. You'll see at the very top there's a pip install fast book but it checks if this is a bash line because it starts with exclamation mark and it's checking whether something called slash content exists that only exists on Colab so that will only run on Colab. So it actually it's importing the fast book and then it's going to try and import it. Okay, great. And we can try running something. Okay, great. So now let's try pushing that back to our repo. So if we now CD into fast book and we do a get status get status changes. It looks like you haven't saved that. Yeah, I thought I did but apparently I didn't. Oh, because it's on a Mac and you have to press command s and I'm just not used to using a Mac yet. Okay, so it tells us that we've made a change. So to remind you the kind of the shortcut for adding something to a commit and giving it a commit message is get minus a to add it and M to provide a message and then you need to say what it was going to do. So I'm going to commit and the message has to be in single quotes so it knows that's all one string test making a change. Okay. So here's another thing that we're going to want to add to our persistent stuff is who am I? So let's run the things it says to okay and then what you'll find is that there's now a .config file that's appeared here that contains that information. So hopefully you won't be too surprised to hear that what I want to do is move that into slide. What I want to do is move that into storage and then VIM storage pre run and sim link it in our script. First time around we'll have to do that manually. Okay, and so I rerun the commit by typing control R to search backwards and type commit. Enter. Okay, so that's committed to my local version of the Git repository. So if I control R status you can see that I don't have anything left to commit but it does say that your branch is ahead of master so that means that I haven't pushed it back to the server. So git push will push it back to the server. There we go. So if I now go back to my local version you'll see that this branch is one commit ahead of fast.io master. So I've made a commit that isn't part of fast.io. So, you know, by the way, sometimes like a lot of the time you create a fork and you make changes because you just want to make changes to your version. Sometimes you make a fork and make changes because you actually want to provide those changes back to the original project, you know, you fixed a bug of theirs or added a feature of theirs. And so if I wanted to send my changes back to fast.io because I think they're an improvement to the book then I can say contribute, right? And then that'll say, oh, okay, you can create something called a pull request. And so the pull request, it's going to show you the changes which is I've executed some cells I hadn't executed before and I've added a cell where the source was one plus one and the result was two and then I could create a pull request and that would send something back up to fast.io and I'll show you what that would look like. Obviously never do this unless you are sure you want to because that's just going to annoy the developers. So then what happens is that would appear on my copy of fast book, it would appear in pull requests. And so here's an example. So when you send in a pull request I'll then be able to see what files you changed using this thing called review and B. I'll even be able to see the changes that were made to the notebooks in a graphical view. So I'll show you what that looks like. So there we go. The person's out of these cells and they've provided a description and so then if I wanted to add this back to fast.io I click this button. That's what people talk about when they talk about making a pull request. Anyway, that's not quite what we're doing. We're just keeping our own copy of it as we discussed. Alright, so. Can we have a version of review and B in local machine? Yeah, absolutely review and B is a start up. They're very, very good. You can absolutely so you can totally add it to your own repos just by going to review and B.com. No, I mean in my local machine. No, but you can use something quite similar called NB dime. Which is actually so it doesn't really make sense to like have it on a local machine because like you don't pull request to yourself on your machine. But what you do want to be able to see is to see differences. And so, for example, with mine because I've got NB dime installed. Let's make a change. So let's delete this cell. Save it. And so NB dime is already installed on paper space so I can click here and I can see okay I've made this change as I've deleted one cell. Alright, so let's try running. Oh, something's happening. While this is running, let's talk about what's happening here. So the first thing we've got here is a from blah import blah statement. So let's understand what's happening here. And to do that let's CD into the fast AI repo. Okay, so here's the fast AI repo and the fast AI repo contains a folder called fast AI. And that contains the fast AI library. And in the fast AI library there are some also as well as some Python files as some sub folders. Now in this case we were importing stuff from fast AI dot vision and that tells us that there must be a directory called vision and there is indeed is a directory called all. And then in there finally we're importing from a module called all and that means that there must be a directory called all. Sorry a file called all dot pi and here it is all dot pi. So what happens when we say from blah import blah is it goes through all the dot separated components except for the last one and treats these as directories and so basically what it's doing is it's going to look for a file called fast AI slash vision slash all dot pi. That's how it gets translated. It's all very mechanical. There's nothing magical or weird about it. And so then if we look at that file there it is right and what does this file do well this file is just importing things and so it's importing something from models basics blah blah now when you're inside a library like this you'll see sometimes it uses dot or dot dot prefixes that's going to be replaced with a file called dot slash basics dot pi this will be replaced with a file called dot slash augment dot pi so we should be able to find if I go control split control SP AUG tab yep there is indeed a augment so I've got a split pane here here's my augment file and so when it goes from dot augment import star what does that do if there's a special variable called done to all this when you've got underscore underscore blah underscore underscore we pronounce that done to blah so this is done to all if there's a special variable called done to all that's a list of all of the symbols that it wants to bring in right and so here's a list of all the symbols it wants to bring in so this this this command here will bring in for example a symbol called ran to transform into this file now this file doesn't have something called done to all and so if you don't have a file called done to all then every single thing that is imported or defined in this file will end up being exported from it so since this line here imports something called ran transform and there's no done to all here that means importing all dot pi should also import ran transform and so that means it should be able to type ran t tab and there it is there is a ran transform and if I hit shift enter you can see where it's come from so it's come from fastai.vision.augment.ran transform which remember is translated to fastai slash vision slash augment slash sorry okay this is actually the name of a method so this will become fastai.vision fastai slash vision slash augment dot pi and then we'll find the ran transform class or symbol inside there and so let's find it so if I click on ran transform I'll show you a really nifty trick if you select something in Vim and hit the asterisk key shift 8 it will search for the next place that this word occurs so if I press star asterisk I'll see the next copy of ran transform and so here's the definition of it so that's what it does when you say from import blah if you say import star then that will import everything that's exported so everything in done to all or if there's no done to all everything that's defined or imported so that's why we now have a thing called ran transform available to us yeah if you ever want to know where something's from a lot of people are used to not using a wildcard import so not using star but instead listing specifically exactly all the symbols they import so somebody does that you can scroll back to the top of the file and search through it and try to find it but you don't have to it's much easier than it is with star or necessary to simply type the name and press shift enter and find out where it's from want to get more info about it do the same thing with the question mark and as you can see it will give you the signature the doc string and the full path name of where it's from right and so this is one way if you want to go and look at the source code for this is I could copy that and type colon sp and paste it and now I've got a third split and here's that file but if I just want the source code for that one thing I can just put two question marks and as you can see this gives me the source code for that thing okay so that's what the first line does fastai vision all import star what does the next line do so if I type doc untar data in a fastai library it'll tell me it didn't quite work correctly somehow it'll tell me all the information about it for some reason this one's not getting the usual documentation I would expect to see that's okay you can always click on source to get a link to where it's defined here it is untar data or you can click on show in docs that's interesting so for some reason to help the documents not working right I'll fix that so you can see here's the details here and you can see that there's you know it says here for example see URLs you can click on these things to get more information about them so there's lots of sim links as you can see within the documentation so untar data downloads and extracts URL by default into subdirectories of Tilda so we now know that means your home directory and it returns a path okay so let's see if we can understand what this is doing so what I generally like to do to understand a cell is to run every single line separately so the most important thing to know how to run every single line separately is to know that you can press control shift hyphen it's control even on a Mac not command control shift hyphen to split into two cells at the cursor so step one is to separate this out into separate cells okay and so then you know run each one and see what happens so after I run this one I should be able to look at path okay so it tells me here that it's been stored in slash storage slash data slash etc now that's good news because we know that means it's persistent so if I create a new instance whatever I'm going to have this same thing now downside is if you have a free or cheap account you don't have much space and you might not want all that space being taken up so let's find out how much space is being taken up so let's copy that head up over to our terminal get over there and so remember we can type du minus sh dot 789 meg so that's pretty big so you might not want that to be there which is fine so you can just move it somewhere else you know put it in the home directory or something like that so one interesting question is here is like according to the documentation it was going to extract things by default to sub-directories of tilda slash dot fast.ai but that's not what happened why is that not how it happened well it says it's a wrapper around fastdownload.get so we should probably look up that documentation to find out what's going on so here's fastdownload add here's fastdownload.get this is pretty much what we saw this is using d.get it returns a path by default it goes into base archive which by default is in this case dot fastdownload for fast.ai to change it to fast.ai you can change them by passing fastdownload oh look at this if there's a config file in the base directory then they will be used for fast download now our base directory for fast.ai is dot fast.ai look in the dot fast.ai directory cd tilda slash dot fast.ai there we go there is indeed a config.ini ah so you can see that paper space has actually set things up for us so that by default all of the archives models we create data we download is all going to be put in the persistent storage so that's like that's a good thing unless it's not what you want right so I would say this is another of these things that we probably want to be able to adjust if we want to so how about we move that into slash storage and then as per usual Tim link it back so when we start the machine I don't know if there's a dot fast.ai folder there or not so what we can do is we can say make.minus p basically creates a folder and all of its parent folders and doesn't complain if it's not already there so I use make.minus p to create a dot fast.ai and we will then remove a dot fast.ai now I'll show you a little trick I want to fill out the word fast.ai without typing the whole thing if I hit control p it will fill in the rest of the word p for previous it will fill in the rest of the last word that it can find it starts with those letters so I want to remove fast.ai slash config dot any and then we will link that back again so it's going to be in slash storage so I'll hit control p fill out slash storage and it's going to be called config dot any so I'll press control p and actually if you then hit control x control p it keeps filling in the rest alright so now we're going to have config any file except I don't want to put it in the home directory I want to put it in the dot fast.ai directory we can test that by copying it paste it there we go okay so we've now got a persistent config dot any I think in my case what I want to do and this might not be bad for most people is I probably don't want the archives to be stored in my storage directory because I'm not going to need them again once it's been unarchived and so let's cd to slash storage slash archive yeah I don't want this there so let's just remove it cool okay so this is now going to put by default stuff that I download will be in my storage I'll be able to use it anywhere but you know I can always move it somewhere else if I want to okay the next two lines kind of go together um which is I'm going to use image data loaders so again you know before you use something it's good to understand what it is so show in docs alright image data loaders is a wrapper around several data loaders with methods for computer vision problems and what you're going to be using is one of the factory methods and all these factory methods accept various things so this tells me that there are various different ways of creating image data loaders and they've got a consistent API so that's good in our case we've got if I hit shift tab I can find out what the parameters are so I've got the path I've got a labeling let's see oh then I've got valid percent and seed so valid percent and seed so what do those mean um so we're using from name punk valid percent is passed to image data loaders from path okay so quite often you know we'll take a argument and just pass it to something else so I have to click on here to find out what that does here we are validation set is a random subset of size validation percent okay no worries optionally created with seed reproducibility um cool and it's got a labeling function so a function that receives a string which is a file name and outputs a label okay so let's just have a look at some of these things so the list of file names so remember the second argument was a list of file names is this so let's pull this out and create something called files so if I press A for append above I'll type files equals pasted here okay and let's look at that okay so files is a list of 7,390 items and it contains various paths in it um I'm going to remind myself go path.base or base path I'll show you what I'm doing on the check let's get rid of all this prefix copy and paste there we go yeah so if you create a special variable called path base path then it will delete that from the start of everything it prints out so it just shows you the relative to here it's easier to read so there's 7,390 things with images slash blah and it looks like it's going to be the name of the breed and then an underscore and then some consecutive number um now this looks a little bit different to what you might be used to seeing um normally if you look at something like a list you expect it to look something more or like this right this is what lists look like in python so to find out why that looks different we can check out what the type of it is by typing type and it turns out it's not a list it's something of type capital L that's a special kind of list which has lots of convenience functions in it we could just use question mark to find out a bit more about it so here's the definition of where it's coming from here's the doc string from it or we could type help L not help doc so help is helps useful as well that's a built into python it shows you a list of all the stuff it can do useful so as you can see it's got lots of functionality in addition to everything that a list does or we could type doc show in docs so you can see this does not just for fast ai libraries it's for all libraries created by fast.ai look pretty similar and so here's lots of information about L let's learn how to read this document this documentation I think we'll stop so L L is like a list okay so a list in python is something that you create like this square brackets and you can print them for example or you can index into them or you can select multiple things from them everything up to but not including element two that's a list okay and L is very similar but it doesn't have any special syntax like python does you have to create it like this but it basically looks much the same as you can see but it has quite a lot more functionality than a list so I don't use a normal python list that often because I use something which is less useful so let's see okay it behaves like a list so I know what a list looks like it's a drop in replacement for a list so when something is described as a drop in replacement anywhere that uses a list you should be able to use this as well so it's got a super set of the functionality it's like numpy it supports advanced indexing okay so what that means is that you should be able to select multiple things from a list at once I'm just trying to remember exactly how we do that I think you can go like this yes okay so I can select the zero thing and the second thing and one go so that's an example and as I mentioned this is similar to what numpy can do so numpy has a thing called an array and so in numpy you can do the same kind of thing but a regular list remember A is our regular list can't do that so you can kind of think of a capital L object as being a bit like a hybrid between a numpy array and a python list you could try to use a numpy array for things that you would otherwise use an L for but the problem is that a numpy array kind of expects everything to be of the same type so you have to be a bit careful sometimes it might do it for us no sometimes it can actually handle it for us it's going to put them into something called an object so yeah I mean actually now I think about it you probably could use an array quite often but they do behave differently actually I'll show you an example so if we go B plus hello then that's what that does in an L it adds hello to the list so notice that L's show you how many items are in it and it also by default doesn't show you all of them but it puts dot dot dot which is both very convenient things a python list oops I used control by mistake works the same way a numpy array doesn't work the same way and that's because numpy is designed mainly for math so if we make all of these into numbers then a numpy array we could do this it adds element-wise okay so you know if you're kind of trying to fully understand each line of code this is the kind of experimentation that you can do and hopefully what you can see here is that all of the information you need to fully understand all of these things is available to you in the documentation which you can link to directly using the dot command and by experimenting but it is like initially it's a lot to learn but the nice thing is that the things you're learning are very reusable right because all this stuff is used all over the place alright so I'm going to wrap it up does anybody have any questions or comments this has been a bit less interactive than usual so I apologize if that's because I'm talking too much but I definitely want to hear your thoughts or questions nothing? that's fine thank you is that because it's too easy or too hard or totally obvious or you want to go away and think about it for me it was very good Jeremy okay great I think it was perfect for me oh cool Jess okay I haven't seen somebody called Jess in a stream before are you a new person here or have you just not noticed you a new old person who just saw it pop up in the forum how did it not pop up for me sooner thank you for joining Jess I am in the Seattle area of USA fantastic well that's great I think yeah okay well that's a good time to finish so hope to see you all see ya awesome see everybody bye thank you