 Hello everyone and welcome to my talk entitled FAST AI NR, Preserving Wildlife with Computer Vision. My name is Andrzej Szczewski and I'm a data scientist working at Absalon. My main focus is on computer vision and I have academic background in theoretical physics. Privately I'm a fan of everything related to winter and because of that I need winters to happen. So I'm very interested in ecology and in particular in wildlife conservation. Today I'll tell you a bit about wildlife conservation efforts and how they can be aided with the help of artificial intelligence. Later I'll move to discussing FAST AI which is a Python library for doing in particular computer vision. And lastly I will tell you why I'm telling you about that in an R conference. So I'll show you that you can actually make use of FAST AI staying within an R environment. So let's jump right into it. Wildlife conservation has recently been transformed by the use of camera traps. So camera traps are devices which look like they're shown on the second image from the left here installed in Central Africa. And they typically are equipped with a motion sensor or heat sensor which will then trigger the camera whenever there is an animal passing in front of it. In this way vast and fascinating data sets are produced images from which you can see on this slide. I would like to point your attention especially to the elusive African golden cat visible on the image on the right, which is the least studied large wildcat in Africa. But actually one of the key problems with using this technology is that most of the data sets look actually like that. So they're almost all empty. And then there are just a few images which contain or among the thousands of images which are empty, you have some images which contain the precious information. So the wildlife conservationists need to sift through all those images to actually find those which are interesting. And this is a problem because it's a waste of their precious time. And therefore one of the possible ways of using computer vision to help wildlife conservation efforts is to automate the task for them. So to build a model which will identify that there is an animal and what animal is in a given image. So let me take you through two examples of such cases. So in the top one, this is Hakuna Madata competition organized earlier this year by Driven Data. Participants were given a data set of over six million images from African savannas. And when joining this competition we have decided not to go for the typical approach taken in competitions of that sort, which is to build heavily assembled large models which are super complicated and give you a slight benefit in terms of your standings on the leaderboard but are not really directly applicable in real life. So we've decided to go for a robust approach. We've actually based our solution on a single network architecture. And using a few neat tricks here and there, we were able to reach the fifth place out of over 800 participants in that competition. Now, after we've developed this approach, we decided, okay, let's now test it in a real life wildlife conservation context. So in collaboration with experts from University of Stirling and National Parks Agency in Gabon, we are working on a data set of over one million images taken in African forests this time. And using similar robust approach, we were actually able to achieve very promising results which are a gift of promise of automating large parts of the work of the conservationists and therefore the results will be published soon in form of a paper. Now, I would like to underline that the two cases are quite different. So in the top one, you see an image of a savanna and most of the images in this data set consists of half of the images taken by the grass, half of the image by the sky and then there are some animals somewhere that you need to identify. Now, in the images from forests, there's a lot more going on. And to tell you exactly how much more, I'll show you an example. So here on the left, you see an image from the forest data set. And our model is actually very certain that there is an kulengu bird in this image. Now, can you see the bird? Well, I could not initially, but after trusting in the model, I've looked through it very, very closely, analyzed each piece of it. And then I noticed that there is indeed a bird here facing with its beak towards the left. It's hidden behind the branches, behind some leaves. Actually, at the bottom of this shape that I've highlighted, you can see a piece of the red leg characteristic to the kulengu bird. So this exemplifies how hard a task the conservationists have. They need to go through thousands of images and quickly identify not only where is the given animal, but also they're looking at the same time for multiple different types of animals. And now, a second example, which I would like to share with you, is on the right. So this is one of the images taken by a camera trap at night. And you can imagine that, of course, the camera is also triggered at night. And they produce images which mostly look something like that. Now, we noticed a phenomenon which turned out to be very interesting, namely that in some of those seemingly empty images, our models are very confident that they see animals. So in this particular one, our model was super confident that there is an elephant. Well, it's impossible to see the elephant with naked eye, so I've looked at the image with some image processing software. And only after tweaking the brightness on this image, so basically just shifting the threshold of white very close to the dark colors, I see that indeed there are tiny pixel intensity differences right in the middle of the image, revealing that there is an elephant family walking right in front of the camera just in the dark. So having those state-of-the-art results in computer vision was taking this robust approach was possible because we've used a library called Fast.ai, which is a Python library, which enables you to, on one hand, which I hope to argue, achieve state-of-the-art results, but on the other hand, it's fast, as the name suggests. So let's now shift gears and focus on that library. So Fast.ai is a wrapper of PyTorch, which is an environment for deep learning in Python. Now, I would like you to take you through an R notebook, which I've prepared. In this notebook, we will execute training in a simple case of a model in Fast.ai, but we'll do that from contacting a kernel or addressing the Python kernel from within R using the reticulate package. So the configuration of the Python kernel is relatively simple. Here is a configuration of a Conda environment. Basically, I've just installed PyTorch, Fast.ai, and a helper library for dealing with images. And then I tell R kernel to use that Python environment for executing Pythonic commands. And I'm heavily using PyRun string in this notebook. So basically, now let's jump into the Python code, and I will show you how fast Fast.ai is in development. So here we load the library and a few other libraries that we will need to process the files. Then we load a data frame, which consists of the data. In our case, we will be dealing with a tiny data set of 80 images of crocuses, 80 images of lily valleys, and 80 images of daisies. So altogether, 240 images. We have their labels in this data frame, and we have their groups, so they're either in the validation set or the training set. Now, in these three lines of code, we actually build an object called Databunch, which is a Pythonic object used by Fast.ai to address data. And here we basically just do the default things apart from specifying where exactly is the information about the validation set, where exactly is the information about the label, and where exactly are the images. Now, with that in hand, we can define a learner object, which is this learn, which the Python variable learn holds. And it's a convolutional neural network learner, which is based on that data, that Databunch that was built before, and using a squeeze net architecture and pre-trained weights for this model. It's a special architecture, which is meant to give you good results with outstanding size. So it's actually very small, and then therefore very quick. I'm doing this because it allows me to train this network on my laptop without access to GPU. It takes a few minutes to train, so it's just a bit too long to show you that live in this presentation, but it's definitely enough to do some basic experimentation. And I also specify that I want to monitor the accuracy of this model during training. Now, I'm using another neat trick from Fast.ai, which is the other find function, which allows me to basically plot this graph, from which, using some heuristics, I can identify what is the optimal learning rate, which I should use for this phase of training. But the key part happens here, where what I do is I fit the model to the data. The only parameters I specify is for how many epochs I should train and what learning rate I should use. And please note that I'm using one-cycle policy training, which is actually a very recent development in computer vision, and it's already implemented in Fast.ai. So with the execution of that line, I'm getting the training to be running, and we are reaching some accuracy of 92-ish percent, which means that if we look at the confusion matrix here, almost all of the images from the validation set are actually identified correctly. Just the three of them are misclassified. Now, that's all fine. We see that Fast.ai can give you state-of-the-art results. We see that experimenting in Fast.ai, building your first model is super quick. Can we do that in R? So let's now go to another notebook, in which I will redo all the steps I did before, but now all the objects will actually be R objects. Again, I will be using the reticulate package, and now a few more packages because I need to do all the processing in R. And after loading the environment, what I do is I load the module Fast.aiVision, but this time to an R object which holds the module. Then I construct this data frame. Again, now it's an R data frame, which contains all the information that is needed for Fast.ai model. Then I construct the data bunch. Again, this data object is an R variable this time, holding the Fast.ai's data bunch. Now I've defined it using R syntax, as you can see. And then I construct a learner, which is basically a variable holding the model, again trained with the same architecture on the same pre-trained weights, and then I'm monitoring accuracy. Here you see an example of how you can use the same trick for finding the learning rate, and after that we execute the fitting. This all happened in R, and we see the result. Now the model is slightly better, because it actually classified all the images and the validation set correctly, which is a definite proof that R is better than Python. Now the learning or the message from this notebook is that you can redo all that we did in Fast.ai in R. So I've trained the model from scratch in R. Now, one other use case that I think is very valuable and that I want to share with you is the following. So imagine that you've heard that there is a very nice model. It happens to be trained in Fast.ai and you would really like to use it in your R Shiny app. How can you do that? Well, I've just told you about quite an awesome model, the Hakuna Madata model that I've presented. Now let's say you've downloaded it and then you actually want to use it. So here is the model. Now, we are again building everything in R, so these are all R objects. We are building a list of images we want to test the model on or infer from the model on those images. We're taking five images from some folder and then we are loading the learner just like we did before, but now we're using the weights that we've downloaded from the internet. We can see that it's already working. We can see how many classes the model is ready to classify. These are the classes from Hakuna Madata. And then we run the predictions. So we run inference from that model on the data that we've prepared. And we get an R object called PREDS, which actually now holds all the information. So basically, for each of the images, we see the likelihood that it belongs to that particular category. For example, the likelihood that the second image is baboon is given by this number. And now we can process this in all the R way we want. So here is a way of identifying, for example, for each image what is the class which has the highest likelihood or the highest score and what the score is. And then we can also, for example, display the results using the R way of doing those things, as you can see here. But also you can embed it, for example, in your R Shiny application. So with that, I would like to go to my key message. So basically, what I've shown you is that AI applications are a reality and AI can also be applied in wildlife conservation. I've shown you that fast AI can give you state-of-the-art results and it has a very low threshold of entry. You can start experimenting with modeling even if you're not an expert in computer vision and only later make your model more sophisticated and better suited to what you want to do with it. And the best thing about this is that you can do all of that not leaving your comfortable R sofa. Thank you very much. And in case you have any questions or comments, please feel free to reach out. Thank you.