 Today I'm going to talk about using machine learning models in JavaScript application. And I chose the name JavaScript because JavaScript can run on server, client, so many places. And basically, that's what we're going to see today. I'm from Mexico. Yes, I have jet lag right now. And I've been a software developer for eight years now. I like to view the community in my city. I'm a woman take maker from Google. And I'm a curious developer. I'm not an expert in AI, but this is how I wanted to know. I wanted to know what was going on. AI is not new. It's been around since a long time ago. But there's a lot of hype right now. And there's a good reason to be hype right now. But there's been hype since forever. So if you think you're late to the game, you're not. This is things that are evolving constantly. So I hope with this talk, you can get your JavaScript skills if you want to start building things or just know what's going on in the JavaScript ecosystem. So for the agenda for today, just want to give some basic idea of what it's AI, machine learning, and then why do we think of machine learning with JavaScript and using TensorFlow.js? We're going to talk about what are models and what are TensorFlow models. I'm going to show you a little demo using node in TensorFlow.js and some use cases that we could start using them. And finally, what's new? Maybe some of you already know TensorFlow and this talk. You can find it on many places. But this is for people who haven't heard it, people who already know it, but want to know what's new. So I hope you enjoy this talk. And so what is AI or artificial intelligence? This is basically a science discipline that does the theory and methods to build machines that think and act like humans. So this is basically the whole idea of why we start programming, trying to build machines that are intelligent. And this is coming from the 1950s. So as I mentioned, it's nothing new. But we're in the verge of an ecosystem that can make it grow so fast right now. And so today, we're going to talk about machine learning. There's also other concepts like deep learning. And we're not going to cover them today, but deep learning covers more important complex problems like generative AI or self-driving cars. And the difference might be basically the neural networks that they use that they emulate the biology of how we think. And so for anybody who has coded before, we usually have requirements. And then we can have data that could be a database, an image, text, whatever. And then we have to code it to do what we want. That is how we usually program. And with machine learning, it's different because we have algorithms which we'll train to make them. So we'll give the same data that can come from a database, an image, text, audio. And we need to convert them first to numbers because the models and the algorithms read numbers. And then we can label them so the machine will try, or the model, will try to find patterns to know how they can feel that requirement. That is the main difference of how we usually program. We know how to code it, but there are complex problems that we cannot do the if, else, infinite option. So that's why machine learning comes into play. Machine learning doesn't need to be added to every application. But there's complex problems that, as programming with just basic JavaScript or no, couldn't be covered. So this is where it comes in. And so this is just the basic line. Just making sure we're on the same page. And then, OK, so why with JavaScript? So most of the machine learning or deep learning is done in Python because it has a very big ecosystem community researchers. That's good. But also JavaScript has a lot of community. JavaScript in Stack Overflow. The survey of this year, it was the 11th year with the most developers building with JavaScript. So we have a lot of people doing web or just doing software with JavaScript. And this is a great bridge so we can build more solutions. If only a few people build these solutions, we won't cover the whole world. So this is an effort that Google is doing. That's why it's providing TensorFlow.js because that's a way to reach more people. And so more people can contribute to provide these solutions. So as I mentioned, JavaScript can run on the web. If you open a browser and a console, you will have JavaScript there. If you want desktop application, you can use Electron. If you want the server, we have Node. If you want to build upon mobile, we have React Native or Progressive Web Apps. And then if you want to run an IoT, you could use Raspberry Pi and put it on Node. So as you can see, you could reach more places with JavaScript, something that Python cannot provide unless you provide an API, of course, but this is running exclusively those models on these devices. And so why TensorFlow? OK, we know JavaScript is a good language, but why TensorFlow.js? It's backed up by Google. They have updates every year. They're definitely building upon this product. They feel there is a future on this. They definitely have a lot of advocates to provide the knowledge so people can start building this. This library has been since 2018, and it has a great community. It's open source, so it's a great way to start building models with TensorFlow. And so it encapsulates the complexity. What does this mean? Basically, there's three layers within this library. The first one is the models. There's already trained models that you can plug and play within your applications. There's another layer, which is the Layers API. This is to create models from scratch. So for example, this is something that if you want transfer learning, it's a way of, if I have a model but I need to modify it for my use case, I'll use the Layers API. And then there's another layer, which is the CoreOps API. This is more for linear algebra. I'm not saying that don't use it. It would definitely be something that we will need to dive in to understand it better. But if you want to get started, Layers API and the models layer would be a great place for you to get started. So what can I do with TensorFlow library? I can create from scratch a model. I can retrain or train a model. If I already have it and I want to train it on this library, I can do it. And I can just run them. I can just put them on the device that I have it and it would run. It has a great ecosystem from MPM. Every model, it's an MPM package. It can be with JavaScript, it can be with Node. So that's a great ecosystem. Another thing, this is exclusively if you run these models on the browser. So we're dealing with a lot of information. Let's say your application needs to read a password. So if you read that password, usually most of the models run on the server. So you will need to move that image to the model running on the server and it can get compromised. But if you have that one, just reading that document on your browser, it's not sending it anywhere. So just read it and infer. You know, yeah, it's okay. Or whatever classification you use for that model. If it's a diagnosis, that's also very important data and very private. So it's another great thing. And or just reading any legal document, talking about privacy. This is very important for us to manipulate. So it's a win if you run it on the browser. The next one is the latency. Yes, we have TensorFlow on the server. And as I mentioned, most of the models run on the server. But if you run it on your browser or on your mobile or on your IoT device, you have the access to webcam. You have the access of a microphone or whatever device, if it has a sensor, you have that available. What does this brings? This brings more solutions, more creative. There's actually a lot of demos out there from creative people because they have access. You know, it's easier to grab a phone or something instead of just your laptop. So this creates more creative solutions. So this is a great, I'd say, benefit of using it. This is on the browser. Less cost. So well, you don't have to have one computer running that server model. Or you don't need to pay it if you're paying for it. If it's from the browser, it's already there. You just have the webpage, right? So it's not that expensive because actually running those is expensive. And the benefit that it can be offline, if you have it on your browser, if your page already loaded, you have access to that model. So if you have low connectivity or in a place where you might need it to use it offline, once the application is loaded, you can have access to that model. It doesn't need to go to the server to run any inference. So these are the benefits of running TensorFlow.js. And so I talk a lot about models, but what are models? So as I mentioned, models can be run with TensorFlow.js. Models, you can retrain them if you need to add something for your specific case. Just write them from scratch. TensorFlow has a big ecosystem in the library, which means that you can have models from TensorFlow, models from media pipe, and all of this can get translated to TensorFlow.js. So it's the only library that can digest other models. So that's why we have a broad variety of models that we could use or translate to deploy them on the browser or on the server. And how do you train this model? So on machine learning, there's very popular ways to train models. The main ones are supervised learning, unsupervised learning, and there's another one which is reinforcement learning. But today we're gonna just look at this too. And from supervised learning, remember I mentioned, you need to give data in the label. So in regression, for example, the most common example you might have already heard is that if you want to predict the cost of a house and if you have that in a CSV, with all the real estate data, that a house with three rooms and two floors can cost this and you have all of those detailed and you feed that to the model, it would predict the number, right? Because you're giving the labels of the features and then it would predict the number. That is regression. The difference between that one and classification is because in classification, you give options. You're saying, is it spam, for example, for an email or is it not spam? So you feed it with details, labels to the model and the model will predict either a classification if it's a classification problem or if it's a regression problem, just the number. That's the only difference. And then on unsupervised learning, this one is very commonly used because you have a lot of data. We live in a world where there's a lot of data right now online and what can we predict from this? So on unsupervised learning, you're just filling the data and there's a way to know anomaly detection. So this is very common for fraud. You may have some transactions that run in the same place and something that doesn't match or it's not from the, if you graphed it, it would be out of place. So that's a way to detect fraud. And the other one is clustering, the same thing. You field data and this is very commonly used in e-commerce. So for example, you can find that people that like copy buy all of these things. People that have baby buy all of these things. And you will see it graph within clusters. That's how you can differentiate them. And this is unsupervised. As I mentioned, you don't give a label, you just give data. So models. What models do we have available in TensorFlow? We have vision, we have one that is image classification, object detection, the difference between them. The object detection can detect where and that classification would only let you know dog, cat or whatever. And the object would give you where and how many. The human body. There's a lot of cool demos out there for people who wanna work out and wanna do the, in pandemic, you couldn't go to a gym. So a lot of trainers created solutions where you could provide the right position of the body when you're working out. So as I mentioned, a lot of people that were programmers or didn't care about it, needed solutions. And that's why there's a lot of demos I guess that we think pandemic, a lot of people wanted to learn. But yeah, we have body segmentation, past estimation, face and hand post estimation. This is used with hand language. So there's, I mean, great solutions that you can build with these models available to you just by plugging in play. Text, sound and others. So these are some of the already available for you. And so what is the workflow that you use for creating a solution? So first, you'll need to think what do you wanna create? And with this, you'll need to find the best algorithm or in this case today, we're gonna talk about finding the best model for your use case. And where can I find this one? TensorFlow, as I mentioned, has some models. Hugging Face also has the Transformers API that they use which does a lot of things. It's not the algorithm. They have a model called Transformers. Also runs on JavaScript. So these two are open source and you can find some models or algorithms. Algorithms not models. Then now I know what I need to do. Do I have the right data? There's also open data that you can get from Kagle or from Hugging Face. This is free for you. Getting data is the most complicated part of building the right models. Why? Because with the data that you field that model is how it's gonna work. So we need to be responsible about how we deploy these models with the right data in production, right? I heard in a conference, they mentioned how you train your model is the world that it's gonna work, right? It's only gonna work if you think the world is that way but you need to look further. So there's a lot of data available for you to try in Kagle and in Hugging Face. After that, you have your data, you need to split it. You need to train with a percentage of that data. You train your model and then with other percentage you would verify that it actually works. It should be data that it hasn't seen before so you can actually evaluate if it's working, right? So that's why it's very important to have a lot and enough data. And finally, you'll have your model and you can deploy it to your application and it would run whatever use case, classification, text, segmentation, all those things. So I wanna show you a quick demo. I've already did a lot of wording. Let's go to coding and let me share because I think, let me know if you can see it. Oh, I think I need to get out of here, yep. Is it big enough or should I zoom more? Let me close this. This is a very basic example. Remember, we talk about the prediction of a house. So this is just a script file called train.js. In here, I'm importing TensorFlow with TensorFlow.js node and I have two X and J and Y, sorry. So the first one, let's say is the number of rooms. So if I have one room, the house may cost and then you can multiply it by that. And then if the house has two rooms and so on. So this is how the data simulation would work. You would feeling that data to the model and then we're gonna define the model. So we're gonna declare here a variable called model and then we're gonna say tf.sequential and then we're gonna add a layer to that. If it was a neural network, we will have more layers. But in this case, we're just gonna train it with this basic example with just one D which is just one array, dimension array. And so we're gonna compile it using this, sorry. With an optimizer and with a loss. This is very technical into how you might need to train this data. I don't really wanna go into the details soon don't confuse you more. But usually this is how you would train a very linear regression problem. And then once you compile this, you're basically saying it, what are the algorithms that the data shape of your data that you're filling it in? And then we're gonna use epochs. This is basically the numbers of times that that data it's gonna run through this model. So I'm gonna put it a hundred there. And then the feeding is the place where you actually train this model. So I'm gonna put the X and Ys that we already defined earlier. I'm gonna pass in the epochs. And then I'm gonna, this is gonna be a promise, right? Most of these functions are gonna be promises because once you download the model or once it's finished training, you would have those answers. And then once you finish the training, I'm gonna remember we needed to verify that it's working. So I'm gonna predict as the model to predict how much a house with five rooms would cost. So I'm gonna pass it here. We need to transform this because the model reads data with us in TensorFlow. So you need to translate every input of data, remember I mentioned, you cannot put an image, you need to translate that into numbers. So that's why it's using this one. And then I'm gonna predict here the prediction of how much it might cost. So I'm just gonna run it real quick. Okay, so I'm just gonna use no train. This is already in the script. So I just need to run the script itself. As I mentioned, it would run for a hundred epochs. So as I mentioned, it's gonna run training. The loss is the one that is gonna evaluate. It has to always be going down. So it finished already. So this is so small. So as you can see, the loss is going down starting from the beginning, well, I did it, maybe 50 would have been okay. But if you see the model is training, the model is improving itself and the loss is going down as it goes to the last epoch. There's gonna be, of course, if you do a lot of epochs, the algorithm would just get as good as it is. So you can do it with 500, but if the algorithm can just do certain modifications on it, it's not worthy. So most of this, when you do it, it would be mostly about try and error, how many epochs, how many weights, what algorithm should I use? But in this case, it predicted that it would be 3.2. It was gonna be three. So it's not that bad, but it's not perfect. Because remember, this is predicting. We're always dealing with predictions, not with an answer of how things are. So it wasn't that bad for predicting, but then let's talk about a more visual example. And I'm gonna use a model called MobileNet. This is an image classifier. This is also open source. Let me just go. Okay, so we're gonna do an image classification. And I'm gonna go to the file. As I mentioned, this is training it from scratch. You will need a little more understanding of what loss functions to use, what optimizers to use. But if you want to just plug and play a model that it's already trained, as I mentioned, TensorFlow has a lot. So we're gonna be using the MobileNet. So in this case, I'm gonna, again, require TensorFlow. I'm gonna import it. And then I'm gonna import the model. As I mentioned, there's also a lot of models on NPM, so you just need to install them. And then we're gonna do a file reading. This is something that we'll have on the browser, but we also have disability on node. So I'm gonna use this one to read an image. This is the function to read the file path. All the read it for the file system here. Okay, here is the image classification function that I want to use. I'm gonna run it in the console. And what I'm gonna do is I'm gonna read that image once I already manipulated the image. And I'm gonna use MobileNet.load. As I mentioned, I already imported that, and I'm loading it, and I'm gonna assign it to MobileNet model. Then I'm gonna assign the predictions to another constant. And that is, if you see there's a lot of await, a single wait. So once I have the model loaded, I'm gonna classify the image that I'm passing through the path. There's the path of the image. So I'm gonna use MobileNet.classified and put that image there. And then I'm gonna console load those predictions. Just making a validation that I have the right arguments. And here is the function that I'll be running on the console. I have here two images, one it's of my cat and the other one is of one. I just wanna make sure you guys see it. So you can see there's a cat and there's wine. So when I ask this model to classify... So I'm using node, I'm just running the script, and then I'm passing in the cat, which is the path that I mentioned so the model can read it. And the console would be the classifications of that model. So it says, this is the constraint, right? That MobileNet has some classifications and that's what I attain to. But if you want to add more, you could just pre-transfer learning, right? So it's saying that it could be the probability of it, that it's an Egyptian cat, a tiger cat, a taffy cat, because those are the options that that model has. And I think it kind of worked because it's kind of a cat there. And then let's do the one from the wine. So it says red wine, the probability of 99%. So as I mentioned, these are options that you have available and you can translate that to whatever use case you might have in your application. So maybe this might be new for you, maybe not, but let's see what's actually new. Let me just close this real quick. Okay, so this is the only requirements that I needed for the one of the image classification was just importing those commands for my application. What's new? Google I.O. in May, they provided, so this was, we run all of this in Node. But what's going on on the browser? This is a product from Google. They're actually building a lot of things up because it's their product. And the browser has backends. That means some options available for you to run your models. It has the CPU, which is basically where the device or the model is running. There's WebAssembly. This is perfect for models that are small because maybe you don't want to use WebGL because of the graphic that it might need. But for models that are bigger, you could use WebGL. And they just released WebGPU on your browser. On the most stable Chrome, you could use this one. And how can I translate that to requirements? You can run stable diffusion models. The test that they did, you could generate an image in 10 seconds when it used to take 30 seconds. So this is something that's happening right now. There's a lot of production products right now using browser. Adobe, for example, is using it for object selection within Adobe Photoshop. And that's running on the browser. The benefits of being offline and you can use it. Actually, Google Meet for the body segmentation if you have a background is using the body segmentation model on your browser. So the background can move while you're moving in your meeting. This is the graph that they show on May. If you see, this is from 2019 till this year. Things are going up. There's people using it. This is actual downloads from the NPM package of TensorFlow. The only things that are going down is because people was on holidays. But this is going up and it doesn't look like, I mean, they mentioned Jason Mays is the advocate of this library and it's not going down. This means that there's a lot of potential to use it on the browser, right? So okay, we know browser, but what's going on on the server? Node is fast. And there's actually benchmarks of comparing Python to Node. Node is fast. And actually, hugging phase changed one of their models, the distilled bird, which comes from bird, large language model, but using less tokens. And they actually changed it to Node because it was faster. These are the comparisons that they made. And it took less for this model to run on Node than in Python. So there's benefits and there's use case where we wanna use Node for machine learning. And what are these use case? Basically, these are the most common use case that you'll find, which is running pre-trained models is the easy route, plug and play, and this is what you wanna start with. If you wanna create a community, they did it well. It's really easy to use these models. Maybe you want to retrain it, okay? I know how to use them. I just need to retrain it with the data from a use case. That's the other one. And the third one, which is completely from scratch. This might be on the browser, as I mentioned, or in combination run, if it's a very big model, I can run it on Node. I know JavaScript, and then I can move it to the browser or just put it out there. So we have this capability to do it both ways. With this, JavaScript is not a replacement for Python. This is what I want to end with. It's not a replacement, but it's a great choice if you're starting with. If you are curious to start using and doing machine learning, this is how I started. I'm not an expert, but this is how I started and it was easy to move from not knowing anything about it to actually start building stuff. And all of this, I learned from really amazing people that are also building a lot of resources for you to start with. Some of those are books, some of those are courses. I highly recommend it. You could definitely use deep learning also, but today I'm just talking about machine learning. But yeah, that's it. Thank you. Thank you.