 Hi, everyone. Thank you, everyone, for being here. I'm very happy. If you have been in the previous talk, I'm going to be the old lady who talks about TensorFlow 1.x, but still this will be applicable to Python, to TensorFlow 3. I keep saying Python because it feels like I'm talking about Python 2 while we should be moving to Python 3. So, yeah, nevertheless. So I'm my human F. I'm a PhD student. So we are going to talk about pain. As a PhD student, I know pretty much about pain and how to deal with painful. I'm actually a last-year student, so that means that I have been dealing with pain a lot of time. So if you decide to go to TensorFlow instead of using Kira, Keras, or PyTorch, or Torch, that means then you have been through hell. And you decide, hey, I'm going to build this model. So, TensorFlow is an amazing library because you have tons of ways to do things, but this is the course for you. When everything is possible, the most probable outcome will be chaos. So if you have 100 developers and 20 different ways to do things, how many outcomes do you have? 200. And if these 20 developers will do the same 20 times, you will have 2,000 outcomes. So I do where every responsible developer does. Like, I want to get up and I see, okay, how people who have more models do things. And I found, oh, okay, people use a train function and then they have a class with all the parameters. Oh, how to do the class? Some people build the graph within the class. Some people build the graph in the functions and train. Other people do completely different things, like there's this function and these are the layers and then they loop over the layers and create the graph while they are in this function, create net. So how should I do it? What's the correct way? I need a correct way. There should be a correct way. I'm a little bit OCD, so I want a correct way. And if you think about it, a model is just a thing that takes data, trains itself, evaluates itself and when the model is happy, say, hey, I'm happy, let's predict. So it's a thing that will take train data, train for a while, then evaluate, and then when you go to inference, it will take a new sample and then predict that outcome. It's not that hard. This should be a way, like, I don't care how is your model, how is anything, but this should be clear. When I go back to my code, I should be, no, okay, this function or this class is the train, this function is the model, wherever. And estimators came to help. So estimators became, they were released in 1.3 and became mature in 1.8. They are going to stay in 2, but things in the custom estimators are going to change a little bit. Nevertheless, since we are talking about good practices, this will stay because we still want to be good developers. We are machine learning practitioners. We're still, we are writing code. We are not writing poems. And this is the API of TensorFlow. So on top of the food chain, we are going to have the primary estimator. That's the thing that developers still use all the time because it's robust, it's very tested, so we know that it's correct. Then if you want to go lower, you have an estimator and a Keras model. We have been listening a lot about Keras models and if you want to go back, let's go back to the other talk. Let's talk previous to this one. And then let's say, oh, I actually want my estimator to be completely wildly different. So I have layers. Now in TensorFlow 2, it's in Keras layers, but still we have some way to define layers. And then we have the Python IPA, the C++, and then all the things that talks to the hardware. We don't want to talk to the hardware. Okay. So where is an estimator? An estimator is a complete model. So it was this pink thing that I had before. And an estimator will give us models for train, for predict, and for evaluate. I'm going to repeat this structure a lot because it's actually what we want in a machine learning system. And these are the functions that we need to instantiate. Train, evaluate, and predict. And save model because, of course, you don't want to train your model every single time. I was waiting one week to my model to be trained. And now I want to use it. So I'm going to wait another week. That doesn't make sense. First thing, let's use pre-train or can estimators. This used to be called can. It's like going to the shop and buying a nice tomato soup ready to eat. So can estimators are commonly used architectures. That means if you want to write a linear classifier or linear regressor, it doesn't make sense that you need to write every single time a linear regressor. We all know how this thing is. And it represents a whole model. What kind of can estimators do we have? We have baseline classifiers. So you build your own, your lovely model and you want to test it against a baseline. So you have that. It's classifier and regressor. I'm going to say classifier for simplicity. But we have baseline classifier, linear classifier, deep neural network classifier. So a neural network with layers. Okay. We have found and can estimators are our thing. We want to use that because we are not going to do anything very fancy. We just want to use these kind of estimators. How are we going to do this? So first we need to create a template function, define the model feature column, instantiate this estimator, and then call the method, switch method, try and test and evaluate. That's all the time. So let's start. Let's create an input data. I will talk later about data, data set, but we need an efficient way to feed data to our model. TensorFlow estimators are built for parallelize, for running parallel. So we want to feed things properly. And here, in two lines of code, we have our data set ready to use. Okay. We have the data. And now we have to tell the model which kind of data is this. Like, is this an input? It's a string? It's a float? Where is it? And we simply tell, oh, this feature column is actually a numeric column. So I'm going through all the train keys and tell my model. All these models are actually numeric columns. Okay. So I said, where is the data? I tell what kind of data is this. And now I need to instantiate the estimator. Estimators are a baseline subclass from STDF estimator. And this is a deep neural network estimator, classifier. And it's within estimator. So this is a deep neural network with two hidden units. Each unit has 10 neurons. And then the output is three classes. So it has three neurons at the output. And with two lines of code, actually one line of code, I built a deep neural network. How amazing is that? We all have to agree that this is pretty good. And then we are going to use it. For using it, I just need to call train. For training, I pass the correct data set. This is the iris data set. I don't know anything about botanic. If you give me a plant, I will kill it for sure. But I know if you give me an iris, I will be able to tell you exactly what kind of virus it is. It's a classical machine learning problem where you need to decide which type of virus flower is a flower. So I train my model with inputs from a training data set. I tell how many times do I want to train my model and the steps. And now I'm going to evaluate. So I want to make sure that the model is correctly trained. So another data set is for testing, for development. And I want to see how good it is. And now I'm happy with my model. And I'm going to go to inference. And this is what I say. And this is the three classes of iris. And we have setosa, versicolor, and virginica. And we measure the petal and the sepal length. I don't know really. It's like, why do you know what a petal is? A sepal length. Yeah, whatever. But I know. And I think that most of you, if you do machine learning, you know this thing. So then the only thing that I do need to do is call predict. And I will get the prediction. And then I will figure it out. Since I know the expected, I can say, if it's correct or it's not correct. This will run in parallel for us. This has been handling sessions. This handles threads. We don't need to worry about running key runners and stopping and waiting for stop iterations. If you don't this, you know how painful it used to be. You need to wait until all the threads are dead. But not really. There's a thread that is still alive. So forget about it. TensorFlow estimators will solve it for you. And that's amazing. However, what about if like, okay, I'm a business student. I don't want to use second estimators. And maybe your problem is not this linear regressor problem, then I'm going to simply plug in. So I actually want to like connect a neuron from the input to the out because because I have an attention and I want to, you know, what kind of things like maybe you want to be creative and probably you want to be creative. So what are custom estimators? Custom estimators allows us to be as flexible as we want. On top of practices. So we are bringing all the things that we have built in estimator and account estimator and we bring it here. So when I go back to my code six months later, I don't know to where the custom estimator where the trainees where the model is with kind of layers. I know it will always follow the same structure. And that's good. And the only thing that we need to do is instantiate the TF estimator and write a model. That's easy. So if you want to use a custom estimator, this is how to do it. This is the recipe. And if you see, all the steps are the same. I'm actually not going to talk about how to create a input function, the final model, the model columns, because that's exactly the same way as you used to be. So as you used to do, so you can create your can estimator, test all your pipeline and then set, oh, but actually, I'm going to do my own. And then you create your custom estimator. Creating a custom estimator requires you to write a callback function with a model to use params, the parameters of your configuration. And if you're, you have, this is tricky, I'm always, because I've used it a lot. Like, you can say, oh, I want all my GPU. I don't want to share it. So you can say that in the wrong config parameters. Make sure that when you're using all the GPU, everyone who's in your lab or using the same GPU agrees. Or you're going to have a difficult conversation to handle. Okay, so this callback function is you basically write layers. So you are writing the model, stacking layers, one after each other. As I was saying, this is smooth, but never trust country. Now, it's getting more stable. At the moment, we only have country layers, stable. And I'm passing the features and the params. So here's the problem of sentiment analysis, giving an input sentence. So my feature is a sentence. And the parameters are the vocabulary size, the embedding size, and the size. So instead of writing this crazy lookup function, I'm going to use a layer that is properly tested for me. And then I'm using a convolution on your network. Again, instead of writing a convolution and figuring out that I don't have leaky memory around, I'm going to use layers. Then I have a max pool layer after that. A hidden layer and the output. So this has moved a couple of slides, but we can agree that it's pretty, see? We always know then, okay, I have a model function. This is the callback. I will always come back to here. Okay. So we have all the layers prepared. We are going to say, how do I want to train? I hear use Alarm Optimizer, but you can use whatever Optimizer fits you. Again, Keras Optimizers are there for you. But if you're still in 1.X or you want to do something fancy, use your own training. And here I'm writing directly to the summary. So no more variables than you need to make it global and then write and then remember, you just write to the summary, which is amazing. If you want to train, you have the callback will receive a signal in the mode. If it's trained, you will return estimator span with the train. If it's an evaluate, you won't compute the accuracy of the model and then you return another estimator object and it's the output of this evaluation. So in summary, estimators are a good idea because it allows us to make good decision about their code. It's easier to debug. It's easier to maintain. It's better for everyone, for your mental health. So use it. If we all agree, the estimators are good and they are. If you don't agree, we can discuss it later. So the next thing that you need to do is talk about data sets. So you have built this amazing model and you are using a fit to fit data. How not efficient is that. So if you have this model and it's parallel and can allow multiple runs and crazy things, don't use a dictionary to fit data. It's rude. It's not polite. So we are going to use a data set. How to use a data set is simply you need to import the data. You will manipulate the data. You will create an iterator and then you will consume a data from an iterator. So you don't need to have all your data ready when you are going to start training your model. You just need an iterator. That's amazing. So how to import data. You can import data directly from a generator. So you can delegate the creation of data to another generator. You can use tensor slices, different flavors of tensors or records. And then CS file, it was like temp not booted here because this is not very efficient. But nevertheless, it's an option. If you want, you can use it. Okay. So how we create a data set. I have this example generator function. Then it's a generator. Then it will be generating inputs. And I need to tell TensorFlow A, this input actually has these types and has these shapes. So that way the graph can be created properly. Okay. I have my data ready to use. But I actually want to add one. Or I want to do something. And of course, if I have this super fast thing that generates data, I don't want to apply a common function. I want to apply a map. Then we'll apply a function to all the elements. And then I want to use everything functional. So I want to be efficient. So I can apply a function to all the elements. I can shuffle my data. I can repeat. So once I see my data set once, I can see it again for another iteration. And I can tell the size of the patch. So how many examples do I need to see in order to update the parameters of my TensorFlow model? How to do it? This simple line. How amazing is that? You can concatenate or chain functions. So I said, hey, shuffle my data. And when you finish, repeat it. And the batch size is this batch size. And with this simple line, I'm creating a data set that is efficient for my TensorFlow model and can be paralyzed. Amazing. Great. I have my data ready. I just need to create an iterator. So we have two options here. One is for the iterator. So I'm going to run all the data once. Or an initializable iterator. If you want to do crazy thing, like crazier and more custom, you can use this thing. Normal users will use one-shot iterator. And this is, I said, hey, data set, please give me an iterator. And it gives me an iterator. So how to consume data? Simply call next. And get next will return me a batch of sample data. And I can feed it to the TensorFlow model. Great. So we have finished this part of the journey. We have seen that estimators are a good practical way to handle our models. We have seen that data sets are the proper way to feed data to the model. And yeah, we want to have all these things in place in order to build a proper model. And when you come back to your code, you know where exactly you are. Finally, let's say then you actually heard about Glove or WordPick or VGG. This is very old. AlexNet. I'm going to have a lot of time for questions because I was super fast. Okay. So you have heard about all these models and said, or even Bert, you have heard about Bert and say, should I write 26 layers models with attention or that's going to be very, very tricky? Don't do it. Use TensorFlow Hub. TensorFlow Hub brings all the ideas from continuous integration and software control version to machine learning. Actually, it's a thing that we should be doing and we have not been doing, but we are solving it. So TensorFlow Hub is a place where we can find models that are reusable, stackable and trainable and you can use it in your own pipelines. So let's say I use a very old one. I use a neural network language model. And how do I do that? I say I want an embedding column and this is the model specification. So this URL is the URL of the model. The last number is the version. So let's say the people who created this model updates the model and say, now I want to use the next version. It's essentially changing that. And then I feed that to my estimator. So this is a kind of estimator. I don't even need to do anything crazy. Just two lines of code. I have state of the art models, a state of the art and tested TensorFlow models. So we should be using it. In an ideal world, I am expecting them people will publish their code, their paper and I can use it because if you write science and I cannot replicate, it's not science. It's you're doing kitchen. And I can cook myself. Thank you very much. Sorry. Thanks. It was nice. Now I can finish with a lot of energy. So bits of knowledge then I want you to take away. Estimators are a good idea. Please, please, please, please when you come back to your code don't you shouldn't spend more than five minutes figuring it out where your model is, where the training is. You should be able to know where is everything. Data science allows for building high performance complex pipelines. So if you build this amazing pipeline of your TensorFlow estimator, you don't want to fuel it with this old slow data. You want to fill it fast. And finally, you have a state of the art driving models to use in TensorFlow Hub. It's kind of easy to publish. So if you have invented a really, really nice thing, please share it. The community will be very, very thankful. And now, yes, thank you very much. Thank you, May. So we have time for a few questions. Yeah, sorry, I was super fast. So I hope you get everything. I'm a very hopeful human being and I like to claim to complain about bad things. So... Anybody? Don't be shy. Yeah. So thanks for a really good talk. That was very interesting. And I will go home and write all of my estimators and see everything that's out there in TFM and everything. And so I was just thinking, do you have any kind of best practices for how to curate your model training performance on, like, what are the models? How do you have a local database where you store the, how your models have been training for the different architectures and everything? Or do you have any solution for that? Yeah, that's, that's an excellent question. There is no standard way to do it, so you need to figure it out your own. I can show you how I do it. How I do it is I write TensorFlow summaries and then I will store. So every time around when you create a summary, you can tell I want to save it in that place. So I save it with a timestamp and then I say this timestamp belongs to the model. So I save, like, the layers of the model. So I remember what model I was training there on the summary. So you can write, we haven't talked about TensorFlow board, but TensorFlow board is an amazing tool. So if you have that thing, like, let's say I have, I'm training different architectures with different parameters. So I will have all the parameters, all the parameters in the same tier. And then when you run TensorFlow board, it will load all your architectures. Be careful, because sometimes you let it run with four loops, four, five, four loops, and then it's like, oh, damn, I have 200 architectures and you need to figure it out which one is which. But, yeah, I didn't find a proper way to do it. This is my approach. Thank you for your question. Okay. Anybody else? Come on. We have five minutes. Sorry. Okay, then thank you for the amazing talk.