 Just continue with this. OK, great. Yeah. Any other new faces today? Ali, have you been here before? I don't remember seeing you. Maybe you've been quiet. I've been here. I don't think I've had the camera on. OK, yeah. Nice to meet you. And Nick is back from his tour of tropical islands. Hi, Nick. Yes. Yeah, not so far away, though. Still pretty close. Yeah. Well, when you're in Queensland, tropical islands are easy. By default. Hi, Marie. Nice to see you. Where are you joining us from? Hello. Can you hear me? Hi. Absolutely. Hi. I'm joining from South Bank, Brisbane. Uh-huh. Yeah. I'm at the Children's Health Queensland Hospital building. Ah, yes. OK. Great. Anybody got anything they wanted to talk about or ask about before we dive in? Because I've got some good news, which is I've made things much easier than the horrible direction we went last time. All right. Sounds like I should dive in, then. I know there was, like, as I rather expected, and quite fair enough, some concern on the forum about how complicated and weird this all is, which is partly because we're kind of like jumping into stuff we'd normally do in part two. And so we haven't quite got the, yeah, all the necessary background here. And so, yeah, don't worry if you're feeling it at all. At sea. But it's a good way to, like, I don't know, start to see, like, some stuff you could look into after the course is finished. All right. So let me just hear me. Yes. Sorry, just on mentioning part two, are you planning a part two this year? Planning would be too strong a word, but as much as I ever plan anything. Thinking about? Yes, I would absolutely like to do a part two this year. Awesome. And the fact that you're asking that question means that you are not up to date on our Discord. So you should definitely join our Discord channel because we've actually been talking about doing a conference or an unconference to go with part two at the end of the year in Queensland. And a lot of folks from places that are not in Australia are saying they would be up for coming here for that. You know, partly a kind of a social thing and also trying to do it all in a very COVID safe way of kind of outdoor and masked and people tested ahead of time and stuff. Yeah, so that's, yeah. So nothing's planned like we don't have dates or a syllabus or anything like that, but we absolutely hope to have something awesome maybe towards the end of the year where we can get to, yeah, get to know each other a bit better and get to know fast AI and deep learning a bit better. Yes, can I ask you? Are you going to continue this workflow next week because I know that the class is going to be soon next Tuesday? Yeah, I think so. Like the fact that I'm doing it this week is interesting because, yeah, like it hasn't been, I've got less time to actually work on the course, but I also feel like the stuff we're doing, perhaps the stuff I can use in the next lesson depending on where we get to. So, yeah, so I think we'll do it next week. We'll see how things go if I get really behind, then maybe not, but then I certainly plan to like continue doing them after the last class, you know, until, I don't know, we all know everything, I guess, and then we'll stop at which point there'll be more things to know. So, yeah, OK, yeah. We don't have to stop necessarily, you know, there's just so much to learn. The only problem is that it's obviously a burden on your time, but it's, you know, I enjoy it. Like my issue is what about when we get to the point where there's nothing left to learn, Radik? What do we do then, you know, is there such a point? Sure, there must, you know, there's. But then we start doing it all in a different language, right? We start doing it all in R or Julian. Yeah, all right. C plus plus, that would keep us busy. You know, I think this is my fifth year of doing fast AI courses. And I'm still trying to complete the, you know, part one. Are you still modest at it? No, it's true. All right, let me see. So. Multitask. All right. I am just so happy about how this worked out, to be honest. Although spoiler alert, it didn't turn out to help our score. The score was about the same. So, but I was so happy at how the whole process turned out. But I kind of want to show you how I got there, as well as where we ended up. And yeah, as soon as I kind of like turned off Zoom last time and I went for a walk and then as soon as I went to that, I was like, oh, no, of course, I know how we should do this. And really, so there's quite a few, but you know, we can make this much, much simpler. So let me explain what we're going to try to do. We are going to try to predict two things, the disease and the variety for each image. And the first thing will be to create a pair of data loaders that look like this to each image. They will have connected three things to them, the disease and the type of rice. So this is going to be step one. So let me kind of show you how I got to that step. So that step one that I took was to first of all try to replicate what we've already had before, right? So patty small. But before I used image data loaders, which is like that's the highest level, least flexible function. You know, we can do all the data processing in a single line of code, but only if we want to do something really standard. And trying to predict two things is not standard enough for it. Right? So we now need to go one layer down. And there's like a lot of really good tutorials on Docs.Faster.AI. And because it's been ages since I've done any of this, I'd forgotten entirely how Fast.AI works. So I used them very heavily to remind myself of what's going on. But for example, there is a data block tutorial this pets tutorial is is great at like it goes through all the layers of like different ways of doing things with Fast.AI pre-processing. The Siamese tutorial is another really good one. So these are some of the things I looked at. And the other thing that I looked at was the actual API Docs. So if I click here on data block, this is actually probably what I found the most useful in the end. There's lots of great examples in the documentation. So yeah, as I kind of like, you know how it is to come back to something a couple of years after you built it and you're now kind of the customer of your documentation. And so my experience as a customer of my documentation was I was really delighted by it. So I can definitely suggest checking all that out. So what you can do is before we were using image data loaders from folder. So if we just do the double question mark trick, we can see the source code for it, you know, and it's the normal size of Fast.AI things. It's very small. And you can see that actually all the work's being done by data block. So data block is the like, you know, still high level API but not as high level. It's actually very flexible. And so we're going to step one that I did was to replicate exactly what we had before by using data blocks. And for this, there's actually, you know, so many good examples in the tutorials and in the book, you've seen them all before. We don't need to talk about it too much. You know, we can basically say, okay, for the data block, the input will be an image. The output will be a category. This is just to do disease prediction. The labeling function will be the parent folder, parent label, do a random split. The item and batch transforms, we can copy and paste from what we had before. And that creates a data block. So data loaders is then a data block dot data loaders and you then have to pass in a source. So the source is basically anything that you can iterate through or index into to grab the things that will be passed to these blocks and to this function. So for this, it's going to be a path. And then we also need to get items. And so the one we get a path, we're going to pass that path into the get image files function because that's the thing that returns a list of all of the images in a path. And let's see if that works. How do you know that in the, okay, so the block, you have an image block category block. How do you know that, like, how do you know that the get image files is going to be able to feed both those blocks? So I guess the short answer would be, you know, to read the documentation about those blocks to see what they take and what they do or any of the tutorials that use them. As you can see, they're used all over the place, right? So you can start with this tutorial or this tutorial or this tutorial. So any of those would show you what it does. Or yeah, the actual, let's say this, this is not good documentation. I'd really never bothered to look at this because it's basically all of it. That's not good. No, we should fix that. I guess because there's so many tutorials. I mean, as you can see, like, I guess the reason I never really wrote jock stories is that it's literally a single line of code. So that's like, yeah. So maybe looking at the code is actually interesting in this case. So an image block is something which calls class.create where class is PIL image. It's going to call PIL image.create. So to find out what's actually going to be called by it, you can go PIL image.create. And you can see it's going to get passed a file name, which can be a path or a string or various other things. So get image files. Then obviously you can either run it to see what it comes out with or let's do that so we could just run get image files. Passing in the thing it's going to give in, which is the path. And so as you can see, it's a bunch of paths. And so we could pass one of those, copy that, and it's going to be passed into this function. So we've now just replicated exactly what's happening in the code. Yeah. But for this I generally just look at the tutorials, which tell you what to do. Could you have two different get items that feed different blocks? We're going to come to that. Okay. So park that. So yeah. So a batch transform, like it gets transformed later after reading, right? Yeah. Here we've got the batch transform. Yeah. But in the image block, because right now we have a fill image or something and it needs to become a tensor, right? Yeah. That's right. It gets changed from an integer to a float tensor later on. That's right. Yeah. That's a, yeah, that's a fairly subtle thing, but that's right. The stick that in something image equals and we look at like mp.array image. It's actually stored as bytes or U8 as they call it in PyTorch. So yes, the, this is going to add a batch transform that's going to turn that into a float tensor, which we can see what that's going to look like. We could run it here, I expect to float tensor. The transformation that we're applying is 224, which is like a square image, correct? A 224 by 224. This one here. Yes. Yeah. So this is doing data augmentation of lots of different kinds. So, let's copy doc and paste. For data augmentation, looking at the docs is particularly important because you can see examples of the augmentations it does. So it tells you a list of all the augmentations and how you can change them and here's some examples of what they look like. These augmentations would happen after the int to float tensor, right? Yes, that's correct. And some data augmentations they operate on the entire batch and some operate on a single example. Yeah, that's right. So the ones in batch transforms operate on a whole batch and the ones in item transforms operate on a single item. And so batch transforms operate because they operate on a batch. Before you get there, everything has to be the same shape. So it can be turned into a batch. So this resize resizes everything to the same shape. And then this does these various different types of data augmentation and one of the key pieces of data augmentation it does is to randomly zoom into a subset of the image as you can see in these various examples here. And this data block API can you also use it with data frames where you would be reading your images from a data frame? We are going to do that. We're going to do that in a moment. Yes. Absolutely. So yeah, I'm kind of like skipping over quite a bit of this because it's super well covered in the tutorials. So I don't want to like say stuff that you can very easily read whereas the stuff I'm about to show you isn't as well covered in the tutorials and it's kind of new. But yeah, feel free to keep asking questions about anything you see. So basically, yeah, so all we've done is we've just this is just the same thing that we have in lesson one. And it's doing exactly the same thing as my image data loader stopped from folder but just turned into data block. And so this is what I did just to show you through my process. Step one was to get this working and then I passed that into a learner and I... So let's go copy and I want this to all to run as fast as possible. So I would use the fastest... Let's do this... Do you... When you make this data loader thing do you try to make sure that the shape that it's outputting is what you need for your model or that's later? Well, I generally use models Hamel which don't care what size they get. So yeah, that's one of my tricks. So ResNet 18 is happy with any size. So actually for my testing I'm going to bring this back down to 128 so it's super fast. And so I just want to get the maximum iteration speed here. And so now I can call learn.bit1cycle and let's do one epoch Okay, so this is going to run in under 20 seconds which is kind of what you want, right? You want something that you can test in under about 20 seconds so that way you can just quickly try things and make sure that end to end it's working. Yep, the error rate is down to 30% so that's a good sign. I guess one correlated question is okay, I understand the input size but what about the output size of your data block? You know that this is what you need for that model. That's what I'm saying the model doesn't care the model's happy with any size. I mean the targets or whatever. You hear about the labels? Yeah, the labels. I mean labels don't have sizes the labels are strings. Or just the shape of that like hey, like is it because maybe different models are kind of trying to predict different types of stuff potentially, I don't know. Like some might have shape. A vision learner is I suspect the thing you're kind of asking is the thing that we're going to be covering in a moment. So maybe put that on hold and then tell me if that doesn't make sense. Okay. I have a question. On the data block you randomly select the amount of records or the amount of the batch size that you're going to process. I don't randomly pick the batch size. No, the batch size is actually selected in the data loaders call. And it's 64. 64. So what is the guarantee that that every single one of the images in this particular case will be selected or there's no way to know. Is there any way to know that every single one will be selected? Yes. I mean, well, I mean, yes, except that we're randomly selecting 20% to be our validation set. But every single one will go through the learner. Of the 80% that are in there, everyone will go through our learner because we randomly shuffle them and then we iterate through the whole lot. In a single epoch, the model is guaranteed to see every example that this train just wants. Yeah. Yeah. And that's what this one means. That's what one epoch means is look at everything once. And so if we put two there or look at everything twice, but each time it randomly shuffles it. So it does it in a different random order. A quick question. What I guess this is by torch data loader stuff. But what actually happens for the last batch? The last batch, it depends. And this is actually not by torch data loader. It's actually fast days data loader. So we have our own data loader. Although in the next version, we're likely to replace it with the fast day one. So it depends what drop last is. If drop last is true, then it deletes the last batch. And if it's false, then it includes the last batch. And the reason that's interesting is that the last batch may not be of size 64. Yeah. For the validation set, it always keeps the last batch. And it's super important to shuffle the trendset. The fast AI does it for you. But if you will mess around with the data loaders or do something yourself. If you don't shuffle the trendset, you might get very poor training performance. Even with a lot of... When we used to use Keras, I used to mess all this stuff up all the time. Yeah, trying to get all those details right. It's really annoying. Just to make sure on something you said in next iteration, you're going to replace it with the by torch data loaders? Yeah, probably. Yeah. You said fast AI, so you got confused. Oh, did I? No. That is confusing. Thanks. Okay. So that was my step one, is to just get it working exactly like before. And then I ran it in the background on the same architecture for the same epochs to make sure I got about the same error rate. And I did. So then I was happy that, okay, I'm matching what we had before. So then step two was to try to make it so that the data block spits out three things, which would be one image and two categories. The category of disease and the category of rice type. So to get it to spit out an image in two categories, hopefully you wouldn't be surprised to hear that we just do that. We say we want three blocks and image in two categories. Now, this variety, we did some way of getting that given an image ID. And actually the way I did it was a bit ugly and since then I thought of a better way of doing it, which is what I think we should do is we should create a dict that maps from image ID to variety. And then our function will just be to look that up. So let's call this image to variety equals and it's going to be a dict comprehension. So we're going to loop through the rows in df. Eter items. Now, I always forget what these differences are. Column name, series pair. Returning a tuple with the column name. Okay, that's not what I want. It arose. Yeah, it arose. Index, series. Okay, cool. I think like this it a tuples is the fastest one. But this is not very big. So let's keep it simple. Okay. This is going to iterate over rows and return index class series. Okay, so we don't really care about the index. Another thing we could do is make the image ID the index and then you could actually jump straight into it. But I think I'd rather not use pandas features. I'd rather use more pure Python things because I think that'll make the explanation a little clearer. So we're going to look through. And so what we want is the key will be the rows image ID and the value will be the rows variety. Okay, that looks good. So then there's a couple of ways we could turn this into a function. And I'm just going to show you a little neat trick which is when you go like the let's pick out something. Let's say we're going to grab this one. When you go like this behind the scenes that square bracket thing is actually calling a special magic method in Python called dunder get item which is a function. This is the cool thing about Python. It's so dynamic and flexible like all the syntax sugar is like behind the scenes just calling functions basically. That's exactly the same thing, right? And so that means that this function here image to variety.dunderget item is a function that converts a file name into a variety. So here's the cool thing. Forget why you can pass it an array and it's going to call each of those functions. Which I think is rather nice. So another thing I find helpful okay cool. So when I call that it complains and it says oh get y contains two functions but it should contain one, one for each target. It thinks that there's only one target. Why is that? Well if you think about it we've said there's three blocks but we haven't told it how many of those blocks are for the independent variable and so we have to tell it. And the way we do that is to say the number of inputs equals and so it's one. We have one input and then the rest will be outputs. So when we do that it's now happy. Okay. And personally before I jump to data loaders I first create data sets just to make sure they work. So data sets are where you just grab one thing at a time. There's no mini batches to worry about. So data sets are easier to debug than data loaders. So you can create data sets using exactly the same approach. And so alright so we've got an error. Okay so here's the problem. It tried to look up our function and in fact it's not indexed. It's not passing in the string of the name. It's actually passing in the path. And so that's why we've got a key error. This path does not exist as a key in this dictionary which is quite true. It doesn't. So what I think we should do is fix this up so that we've got train images, bacterial leaf streak, blah blah blah. Here the get files function, the output of that is being passed to the get items is being passed to get y. So get image files. Yeah so we haven't kind of gone into the details of exactly what's going on behind the scenes Hamel. Let's do that in a moment. I kind of like the way you're wanting to jump into the nitty gritty. But it's a little bit It's a problem that I have. It's a little bit more top down, right? So I'm going to get to your bottom up. We'll meet in the middle. Okay, fair enough. By the way, your video is not on. It's fine. I just don't know if it's intentional. I always like to see people when they're you know, seeable. Hello. Okay. So we're not going to use this trick after all. We're going to create a function called get variety. And so it's going to get passed a path. Okay. And so we're going to return image to variety and we're going to return image to variety with the name of the file. So the name of the file is a string. We we need image to variety the dunder thing. Yeah, I'll just grab record actually. Yes. Okay. Oh, and then we need to use that. Okay. Okay. Okay. So dss contains a dot train data set. Okay. And it also contains a dot valid data set. Okay. And so we can look at the zero thing in the training data set, which is a single thing, right? So we can have a look now this image and why one and why two and so then we can look at the image for example. Okay. So what happened here is that get image files returned a list of paths. The first one got passed to image block, which as we saw earlier got passed to pio image dot create. And here it is. And that path name also got passed to a function called parent label. In fact, let's do it, right? So let's say file name equals get image files and then the thing that we passed in, training path and let's just get the zeroth one. Okay. And so there it is. Right. So it ended up calling pio image dot create with that file name. Okay. It also called parent label with that file name. Parent label. Okay. And it also called get variety with that file name. Jeremy, can we look at get variety one more time? I'm just curious how you build the path. I didn't. I removed the path I called dot no. Okay. Okay. Yeah. I see. Yeah. In my original version of this I did it the other way around of like building back up the path and then realize that that was kind of stupid. So yeah, it's unique. So that works. Okay. One question. Okay. Okay. This could be too you know, low level, but just let me know. Can you have multiple get items? Is this the right place to ask that? So it wouldn't make sense to have multiple get items, right? Like every item returns a single thing, but it could be anything you like, right? It could be it could return a top hold or a list or an object or whatever. Right. And so or dict. And then get y and get x and then the things responsible for pulling out the bit that you need to pass to your blocks. Now we don't we don't need to get x because image blocks just take paths directly. So if I needed something a bit more like I needed wanted to put more things and get image file like have it emit a tuple. Yeah. And would I have to like make my own image block to ignore? No, not your image block. You would write your own function just like get image files that returns the list of all of the objects you want, which have all the information you need. Okay. And then like it almost never happens. I don't think that's ever happened to me because like nearly like nearly always there's like a row of a database table or a path or something has all the information you need to like go out and get the stuff with your get x's and get y's. That's like the central piece of information for each row. And based on this information you can read in text, you can read in images, but you know specific to that one row. Actually, let me show you my hacky version because this is the version that uses a data frame. So this is, so the version that uses data frame see. Is it right to think and that's interesting. Let me just do this and let me come to your question. Okay. So in this data block I started out with a data frame like so. Right. And so by passing into data block data loaders I passed in the data frame it's going to get it's going to get each row. Right. And so then get y becomes which is just a function which I mean let's look at it. It doesn't do much. Let's see what it does. So it's done as an object because you can do things like add in a prefix path and a suffix path and you can spit it with a label delimiter and whatever. But basically all it's basically doing okay and it like checks what kind of thing you're passing in but basically all it does is it calls get atra to grab the column and we call reader for under like reading data. Sorry is this called reader function specifically for data frames. I mean it can work with anything basically that you're so what it's doing here is it's yeah it's saying grab that column but it's really I've only really used it for data frames but you could use it for anything. But yeah so basically here get y is saying okay well let's return the index one field and the index two field and what's up with get x? Yeah so because now we're being passed so you can't pass a row of a database table to paoimage.create so get x is this function which basically is going oh it's going to be in the training path slash disease name slash image name so and then there's a special case for the test set because the test set things are not stored in subfolders according to label because we don't know the label so it's just directly in the test path so that's the as I said this was more hacky I don't understand this really helps so like get x is kind of like get y you can have a list in there yeah you can have it yeah it's totally flexible and I mean seriously how more like this like we have so many examples of all of these patterns in the docs in the tutorials so like this exact pattern let's take a look at one right docs.basterd.ai so tutorials let's do data block tutorial right here look mori label so here's one and yeah you can see here this is even splitting based on columns in the database table and here's the call reader using prefix and here's a call reader using a label delimiter and here's the examples coming out yeah so there's yeah lots of examples you can you can check out to see how to do all this yeah so I think I'm at a point now where I actually do want to go into the weeds so Hamel you're now after this totally free to ask any super weedy questions the most basic kind of data block is called the transform block and the transform block basically it's going to store a bunch of things you pass in it's going to store things called type transforms it's going to store things called item transforms it's going to store things called batch transforms and also it always adds one thing which is to tensor because PyTorch is tensors if you look at the image block we saw that that's defined as a transform block where the type transforms is this and the batch transforms is this so now's a good time to talk about how this all works what this does so if I pass in here transform block and don't pass any transforms it won't do anything right so if I let's get rid of like pretty much everything right but let's do that separate cells so it gets a little bit easier to read okay here is the world's simplest data block okay so if we call that as you can see all it does is it takes the output of get image file 0 and turns it into a tuple containing one thing which is the thing itself if we have two transform blocks returns a tuple with two things in it so and the reason it's returning tuples is because this is what we want when we train we have batches right containing inputs and outputs potentially multiple inputs and potentially multiple outputs right so that's why you know indexing into this gives you back a tuple my question yes the block the blocks can either be a list or a tuple I don't know probably okay no idea um okay okay so then we can like do stuff to um the first thing in the tuple so getx equals um let's get a lambda 00 dot name to do with lambda right does name have to be call no maybe it's notebook restart time it might probably not restart time oh I wonder if something happened to my GPU server I mean something has happened to my GPU server really never happened before it looks like it's back oh okay it just recognized it disappeared that's wild it doesn't really matter what are you doing right now I'm just looking at the logs see if anything just happened so you see what happened here is um we you know got the first thing from image files which was this and um getx got its name so we could also do uh get y call slander 00 dot parent say okay so um it first went like first that thing went to the transform block uh dead items yes so whatever dead items got went to transform blocks and then went to getx and gety uh well transform block doesn't do anything right um at all unless you pass the transforms so yeah so it's basically um but the number of them you have is the number of like pipelines it's gonna it's gonna create so if we created another one but generally if you have like an image block it would do something so we're gonna we're gonna get to that yeah so here look we've never we're we're not quite there yet right so let's get to that um and it's not quite the mental model you've got I think now that I've got three transform blocks I only have things to create two of them so it's it's sad right um and so we could um put them here for instance um and that would be the last one is the y and the first two are the x correct unless we say number of inputs equals one right in which case now we get x is just gonna have to return one thing it's gonna be one function and get y will be two things okay um so um you could um you know you could even put it here instead right so you could say oh well this is actually let's move it we could put it here item transforms equals and so the stuff the transform block is stuff that is applied for that transform um how is that not working slightly surprising to me uh okay it's to be a type transform okay type transform so it's now converted to the type it's meant to be so uh how uh radic you were asking about image block I'm just you know curious how all the pieces let me show you let me show you so um let's do it manually so image block is just this okay so let's not use image block let's instead the item transform work let's figure that out later once we figure out what's going on here and then we'll debug it okay so now we've got three transform blocks two of them which do nothing the first one of which is going to call something dot create that was pao image dot create um so transform blocks don't if you look at the code of them transform blocks do anything at all right they actually they only store things there's no there's no done to call there's no forward there's nothing transform blocks don't do anything they just store stuff um the data block is the thing that's then going to go through and say okay for each thing call its type transforms and then call to tensor and then call transform transforms and then data loader time call its batch transforms so does that help answer your question hamel it's not that transform block doesn't get called it just stores the list of things that will get called at each of these times the first thing that gets called is type transforms wait is that right you think that's not correct the first thing that gets called is get X and get Y and then the result of that is passed into type transforms and so get X and get Y so get X would be responsible for making sure that you have a path that you can pass to pao image dot create that's the order so this whole path of what happens you know a sequence that lives in data block that lives in data block exactly now the data block code is frankly hairy and it could do with some simplifying and documenting and refactoring it's not long it's about 50 or 60 lines of code in fact it's almost all here but basically when you call dot data sets really all it's doing is it creates a data sets object passing in all of the type transforms to it and the answer to your question Hamel why didn't the item transforms get done is because item transforms actually get done by the data load or not by the data sets so data sets only use the type transforms and basically the only reason there's like quite a bit of code in here is we try to kind of make sure that if two different things have like the same type transforms we kind of merge them together in a sensible way so there's stuff to try to make sure this all just works I was going to the type transforms are separate from the items transforms because of some optimization you can do with the type transforms because the type transforms are they're happening earlier they're happening before data loaders time so data loaders are the things that are going to take tenses right so or at least things that can be converted into tenses so type transforms are the things that are going to create your data sets for you and they're going to spit out things which need to be convertible into tenses and then data loaders has item transforms which are things like reshaping everything to the same size and batch transforms which are things like data augmentation but you can have an item transform run on the GPU or not on the GPU right it depends on the ordering does it always run? I don't think an item transforms generally going to run on the GPU because it's not a batch yet I mean maybe it's theoretically possible but that would be pretty weird because you need things to be in a batch before the GPU can be optimising it effectively and everything in batch transforms will run on the GPU assuming that you're using a GPU I mean there is a okay this is some part of the codebase we're not looking at today but there is a I can't remember I think it might be a callback which sticks things on the GPU so it just depends on whether things are before or after that callback that's probably a bit of a distraction so let's skip that for now to kind of revise the difference between data set and data loader is it best to revisit the PyTorch documentation and kind of pretty much we have our own implementation of them but our implementation of data loader is a super set of PyTorches and PyTorches data set literally it's an abstract class it doesn't do anything at all so a data set is something that you can index into and it returns a single tuple of your independent independent variables that's what a data set is defined as by PyTorch and therefore that's what we do as well a data loader is you can't index into it the only thing you can do is iterate through it, you can grab the next one and it gives you a mini-batch which is a tensor so that's the difference but yeah that's a PyTorch concept I guess I'm trying to understand the type transform thing why it has to be done in the data set before the data loader it doesn't have to be but it's like we want data sets it's a very convenient thing to have to have something you can go into and grab items numbered that's the basic foundation of the PyTorch data model is that there's things you can index into the type transform aspect of it you need something that converts the output of get image files into what you want in your data set and that thing needs a name and the name we gave it was type transforms I think I understand this is not the only way you could do this but it's our way that's really nice because we now have this thing that you can say like oh Hamel can you show me the 14th image and it's label and you can say yes no problem in Germany you type dss.train bracket 13 and there it is so yeah that's just a convenient thing basically I guess a question around that is that if we did not have type transforms then it would just be one more step in the item transforms right? yeah I think so your data sets would always just return a single thing or maybe two things that get x and get y results and then your data loader would have to do more work basically exactly yeah which would be a perfectly okay way to do things as far as I can tell but I think it would be a little harder to debug and work with and keep things decoupled yeah I think that's a reasonable comment is it like anything you want to do up front that is like kind of uniform across your whole data set maybe put it in the type transform that you don't need to change at training time basically like anything that you want to be able to like index into it and look at that thing really you know if you're not sure where to put it I'd say just chuck it somewhere and don't worry about it you know like you know we kind of put the rule is that you know you need something that can be turned into a tensor like that's the way fastai does it so you need to make sure that your type transform when you're working with fastai returns something that is a tensor or going to be turned into a tensor which PIL image can be for example okay I think I understand it's kind of like you want to make sure it's like a convenient thing that you understand to look at yeah yeah okay so then okay so I can remove all that because this is the definition of image block so let's replace it with the word image block okay and then let's change okay so let's pop a dot name here um here's kind of something we want as our label right that's one of our labels and then the other label we wanted was the function called get variety right now we can't this breaks our rule this can't be turned into a tensor because it's string so um what do we do about that um you might remember from a previous lesson we learnt that what we do is we replace strings with integers where that integer is a lookup into a vocabulary it's a list of all of the possible options so if we change this to category block that is exactly what category block will do right so category block it's got a type transform category which I'm not going to go into because it's not particularly exciting um but if you look up the documentation for categories you can see how it does that so basically internally now you'll find that the vocab is stored for these things so if we look at this at the high level get items by the way here's a vocab right it's got two things it's got the vocab for the diseases and the vocab for the varieties sorry Rado so get items gets us the rows or the examples or whatever allows us to the core for a single example and then from get items we use get y or get x to transform it somehow so that we can pass it into those blocks specifically pass it into the type transforms of those blocks into type transforms and type transforms are things that can get triggered right so they're doing a little bit something similar to get y but like a building on what get y does correct exactly because because these are like very general things right and so I didn't want you guys to have to write your own every time so these these basically say I will work if you can pass me a path to an image and this says I will work if you pass me a string and so get x and get y then are responsible for ensuring that you pass them a path and pass this one a string and get image files is already returning paths so we don't need to get x for this guy but it's not returning strings so we do need to get y for these guys okay so I'm going to finish I'm going to run it slightly over time but let's have a look at so this is exactly okay so this is exactly the same as what we just had right and so then we also then add the two things which is the item transforms and the batch transforms some other time we will talk about how it is that how come this is not being applied to the categories it's only being applied to the images for those of you interested in skipping ahead the secret is using fast cause type dispatch functionality anyway so that's that's why we're getting these three different things the image we've got y1 so Jeremy if we had an image if we had an image block in our for our Ys for our targets then item transform would get applied correct and there's a have a look at the Siamese tutorial on the fast docks because that has two images yeah and like if you think about it anytime we do segmentation that's exactly what's happening right the data augmentations happening to x and y and this is like really unusual I don't know of any other libraries that have this kind of totally transparent ability to do bounding boxes segmentation point clouds whatever as dependent variables and have it all happen in unison very very automatically probably said it used to be maybe there is now okay so so now I can create data loaders from that and thanks to the magic of fast ai this is so cool check this out it's actually auto labeling it with each of our categories so thanks to stuff we'll discuss later basically this stuff called type dispatch fast ai does a lot of things automatically even though I don't think I've ever like explicitly coded this to work it just does because of how the API is designed so we now have something which can create batches of pictures and two different dependent variables each one of which has a category and so what we will get to next time is it actually turns out well briefly mention it now actually all that stuff I did last time about messing around with multiple different heads is actually totally unnecessary all we need to do when we create our vision learner is tell it we don't want 10 outputs but we want 20 outputs so normally it automatically figures out how many outputs you want by how many levels are in your categorical dependent variable but in this case we've got something custom which is we've got a tuple of outputs so we have to tell it we want 20 outputs that's going to make the final matrix that it multiplies by have 20 outputs now then you basically need to tell it what loss function to use and so if you look it up it turns out we used to use a loss function for this called cross entropy loss flat so we're going to call that exact loss function on the first 10 items and we're going to compare that to the disease probabilities and then the second 10 we're going to compare to the variety probabilities and we'll do the same thing for having an error rate which just looks at the first 10 hit the error rate for disease and the same thing for variety look at the second 10 the variety and so basically then if you train that it's going to print out the disease and the variety error and the loss function will be the loss function on both both the two hubs and interestingly for this single model this 2.3% disease error is the best I'd ever got for this architecture so at least for this single model case this was better than training something that only predicts disease anyway we can talk about that more later because we kind of spent more time I have a quick question the last layer it's a flat 20 output layer does this mean at inference time that we would have to do the softmax plus what would it be I had all that for you automatically great and by the way in the inference functions you'll see there's always options as to whether to decode it and whether to put the final activation function on it and stuff like that actually now I think about it in this case because we used a custom loss function I think that would have broken its ability to do it automatically I'm going to say you would need to add a softmax if you wanted to although you actually don't need to because at least for the Kaggle competition I just needed which disease had the highest prediction and whether it's softmax or not it's going to be the same because that's a monotonic function so it depends whether you actually need probabilities or not in my case I didn't have to do it yeah you can probably look at the first 10 or the second you can see it because otherwise so I was using TTA to do test plan augmentation and I stacked up and I did an ensemble of TTA and then I just did an argmax on the first 10 alright alright in the architecture you selected for Resnet 18 128 is there any programmatic way to find out the size or the input size of the models that you are trying to use these models handle any input size alright alright alright all the Resnets and all the ConvNext handle any input size alright thank you that also tripped me up in the beginning but there's a lot of interesting stuff there that might take a whole lecture to understand all that thanks gang see ya thank you