 Thank you all for coming. I'm very excited to talk to you today. I would like to thank Marty and Frances for getting such a big room. It's amazing, so please give them a big round of applause too. So I'm Gilda. I talked here last year and I'm coming back this year. So I work at Le Boncoin in Paris and today I'm going to talk about AI Search with Go and TensorFlow. So, spoiler, AI is really not about intelligence at all. It's more about magic tricks, doing things that you wouldn't expect a computer to do. But nonetheless, it can do a lot of different things. It can make your phone call for you. It can beat multiple pro-gamers at StarCraft at the same time. It can make up, invent some new, some false celebrities, or it can swap faces in a very realistic manner. So today I will show you how you could use this kind of state-of-the-art model into your Go application. All right. So the plan for today, first of all, we'll review a bit some of the basics of AI and deep learning and machine learning, and we'll see how TensorFlow and Go work together. Then we'll see a first concrete example with image classification, and then we'll see how facial recognition can work too. And then we'll see how we can wrap this up to make an image search. And then this will be the conclusion. So AI and TensorFlow. So it's a very good time for us developers regarding AI, because all the big players right now have a huge focus on AI. They are all competing to get as much traction as possible into the AI product, and what the result is is a lot of different frameworks that we can use and that we can do very cool thing with. So Google released TensorFlow, which can be used with Keras, Facebook as PyTorch, Microsoft as the cognitive toolkit, and Amazon developed MXNet, which is used by other companies too. And you can also find very easily some models online. So the same Google, Facebook, Microsoft are giving away a lot of models that are ready to use for you. So let's see what are the basics of AI. So it all starts with one of this little buddy. So this guy is a cell and is getting some float as an input and releasing back a different float as an output. So the function usually looks like something like that, but we don't really need to get into the details. So that's the shape of the Sigma function most of the time. But what is important is that these guys can combine with some other ones and start to make interesting things. So after a while, you can have some nice things happening. For example, from a non-obvious picture, you can guess the breed of the dog. Maybe you can fail in some other example. It can turn a recent float into Nicolas Cage, or it can protect you from a non-safer work representation. So I want just to clarify a few terms that will certainly pop up later. So architecture is the shape of a network. So that's a very important factor in a network. And the next one, the one I will use the most is the model. So a model is basically an architecture with all the weights and bias defined. So here we see the function we've seen just before. And the pre-trained model is the kind of model that you can get from Facebook or Google that already do a function well. And then a saved model is a format to export this model and to share it with some other people. So now TensorFlow is a framework for creating, training, predicting, exporting, and importing neural networks. So it's a C++ core. Most of the time for the training part, it's used using its Python binding to its C API. And so that's using Python, you will do the, you will create the network, you will train it, and then you can export it to a saved model. And the part that interests us is the, on the other side, once the model is trained, you can import it into Go, and you can run the prediction using the Go API. So in that talk, we'll really just focus on the Go part. So now let's see how the code looks like. So here is all the code you'll need. This is all the specific TensorFlow code. So it's split in two, in two different parts. So firstly, you load the model, and you prepare the input that you want to give into that network. And then the second part is actually running the session, giving the feeds, and getting the fetches at the other part. So we can split this into three parts. First one is getting the model, getting the model and loading it into TensorFlow. The second part is building an input, and then filling it into the input of the network. And then the last part is to fetch the result and to interpret it. So let's see a concrete example with image classification. So image classification... Well, image classification is basically taking an image and extracting some labels. So there was a beautiful cat, Siamese cat, there, with some scores about the fact that it's a Siamese cat. All right. So one of the most common database for this is ImageNet. Well, some of them are loaded, but not all of them. Well, just two seconds now. I'll move on to my mobile thing then. All right. Is it better? No? Not really? Yeah, yeah. Okay. Well, anyway, so basically you have an image of a cat at the input, and then you get some labels. So we'll need to add three more steps before actually running it into Go. The first one is to find the model. The next is to run it into Python and then saving the model. And then the two other ones we know about. Yeah, it's better. All right. So to find a model, there is a website I like a lot. It's called Model Depot. There's not too many models there, but they are very well documented. Another good resource for this is the one from Google, Facebook, and Microsoft. Most of these models are on GitHub, too, so you'll find a lot of interesting things. And please don't be afraid to look at some research, too. They often come with some pre-trained models. So I chose that one, which is a quite simple one and a quite light one. So now I have a model, I can download it from it. It's actually MIT licensed, so it's nice. So we'll run it into Python. First of all, we need a few imports. So this is based on Keras, so it's mostly Keras imports. And now here is the Python code. So it's quite simple, a bit like in the Go code. The first part is loading the model, then we format the input, and then we render prediction. All right. So now we can run it into Python, and we can actually have the correct labels. So the next step is to save it into an export model. So we'll add a few, a couple of more imports. And basically the only thing you need to do is just to surround your code with a few more lines to connect your session to TensorFlow and then to export it into a saved model. So now we have a folder with everything we need to use our saved model. So now one step that is usually simple is to find the input and output layer names. So if we're here with using Keras, so we can just print them straight out of the model. It's one of the functions of the model. If you're not using Keras and if it's not documented, maybe you'll need to print all the operations and to just look at all the names and find which one looks the most promising one. Or the last solution is to debug the Python code. So in our case we're using the first option and our input model is input underscore one and the output is prediction softmax. Now we need to format the input image. So here we can see the shape of the input layer. So it's a 224 times 234 image with three channels as RGB. So there's just one more function we need to apply it to these channels. It's just a simple function. Actually it's just a mapping between a smaller float range. So this is what we have and we do it for every pixel. So this is how the go code looks for that. So it's only using the standard library, the standard image library, and then creating a tensor flow tensor. All right, so now we have our tensor that we can feed into our network. Now the last part is to interpret the results. So the shape of the result is a slice of a thousand floats and this actually corresponds to the thousand classes there is in ImageNet. So it's basically all the objects that we have and they are linked to a score. So usually what you'll do is to keep the 10 best results maybe. So perfect. Now we can get the output and we can find out it's the same as cat just the same way as we did with the Python code. So to wrap it up, so now you can give any image from any source you want from your go program and we can get the 10 best labels. So face recognition. I won't go into the code for face recognition but I just want to give you the basic of how it's working. So the first step is to detect the faces. So it takes a picture of any size as an input and it will give you back some some boxes and scores about the detections. So in the exact example we can extract five different faces. The next part is landmark extraction. So the shape of the input shape is also a square image of 112 pixel wide and the output are 68 landmark points and the maps to some peculiar landmarks into the face and you can use it to straighten the face to improve the performance greatly. And then the last part is the descriptor extraction. So again it takes an input an image as an input and the output is a 128 size slice of float which actually represents a coordinate in a 128 dimension space. And what is good about that is that you can apply some acridian distance with it. So the acridian distance is the most common distance and and so this define some distance between faces. So the smaller the distance between the faces the more likely it is that it's the same person. And yeah it's especially nice for us because it's a very lightweight representation. It's very fast so it's good for search. The models I've used are from Face API.js which is using TensorFlow.js. It's a very nice one because it's only using TensorFlow as a dependency and I really wanted to keep it only TensorFlow and not using OpenCV or Delib. So I had to do a bit of extra work on this one because the code was in JS so I had to translate all the code from JS to Python which was quite simple because it's almost the same function and then I was able to save the model and to load it into my Go application. So search is less than as an exercise to the reader but it's actually quite simple now. Now that we can label our photos it's quite easy to just return the matching photos when the user is querying with a keyword. And about the faces. Once you have some distance between some of the faces in your database when a new faces is found you can just calculate the distance with all the other faces and when you find a face that is close enough then you can tell it's the same people. All right so the conclusion. Yeah first of all I want to point you out to my repo where you can find all the models I've talked about. Yeah you can find it quite easily I think. What I want you to do is just to try it. I made a nice docker image that can run just to try out to see the performance of this algorithm. Or you can use it as a library. It's hopefully simple and ready to use but please feel free to contact me if it's not working or if you would like something different. And also I would like to encourage you to try new models. So as we've seen the TensorFlow and Keras models are very easy to integrate into your applications and yeah you should try. It's nice. Other models, other frameworks will most likely require some conversion which is still experimental and it will result in some significant extra work. And so yeah remember the five steps to use a new model. Finding the model. Finding the model is just searching on Google so that's the kind of thing that you're already doing. Running into Python usually it's not a problem because most of these models are well documented. Saving the model is just a few lines that's simple. Maybe most of the time you'll have some troubles formatting the input. If the input is an image it's almost always the same kind of formatting so it shouldn't be too much of a problem but if you start to play with sound or this kind of thing maybe you'll spend some time because there is not much go code doing that. And interpreting the result can be can be very simple for some time and a bit harder at some other points. But yeah the last result is always to just render the Python code step by step and to see how it's done in Python. So it's not impossible to do. You should definitely try it. All right thank you that was all I had to show you today. If you have some questions. Yeah we have some time for Q&A so if you have questions raise your hand and if you want to leave please do so silently thank you. Any questions? Okay so just round of applause thank you very much.