 Closing up into the mic. There we go. That's better. All right. Good. Everyone can hear me in the back. Cool Let's get started Hi, if you don't know me yet, you do now. My name is Gabe Hollenby. I am a technical evangelist I work for AWS. Who doesn't know what AWS is? Do I need to spend any time explaining AWS to any of you? Perfect. Cool Who doesn't know what a tech evangelist? Cool I'm a software engineer who has the very lucky job of getting to stand up in front of all of you lovely people at places like this and show you how to build cool stuff with AWS So I'm a builder. I try and write code every day I don't always get to but I'm technical all my background is in software engineering But now I don't write production code anymore. I build cool demos and get people excited about, you know What AWS has to offer? That's my job. If you want to keep in touch Twitter is the best way. That's my handle and I would love to hear from any of you on what you think of The presentation today. Anything you think I can make clear for future times when I deliver this content I really value that feedback especially. So don't be shy All right What we're going to cover today is a few things I'm going to try and speed it up a little bit because I want to be respectful of your time I've done this talk in the past and it runs about 40 minutes But I think what I'm going to do for this one is tone down the AWS stuff a bit because that's not what's super exciting to you Well, nobody's here because they wanted to I don't think because I'm here I'm going to spend as much time as I can showing Python code and talking about how we're going to do some deep learning using Python It just happens to run on AWS I'm going to kick off a demo pretty early on because I want to do a live demo I think it's more fun that way if it doesn't work. I got a backup, but let's do a live demo first It's going to take a little bit of time. What I'm going to do is train a custom image classifier During this talk in the background and it doesn't take that Actually that long to train takes a few minutes to train takes a few minutes to actually deploy that to an up An endpoint that I can then pass images to for like live inference And so I rather than waiting at that point and watching progress bars go by I'm going to kick that off basically by running a bunch of code in a Jupyter notebook I'll then I'll explain some concepts. So then we'll come back to the deck I'll explain what the heck is going on and then I'll review the code with you guys So it's not just like oh wow Gabe just ran a notebook and they had this That's not the point of this talk. The point is to show you how you can do this yourself Okay So What is deep learning? I'll just set the stage before we we go and we gather our our data for a classifier So deep learning it falls into the realm of machine learning. What is machine learning? It's simply a way of having computers learn how to do things that are Easy for us to do as humans but hard for us as programmers to explicitly program into computers, right? Here's some common examples self-driving cars sentiment analysis object detection facial recognition, right? These are popular Applications of machine learning and specifically of deep learning and I'll get into what deep learning is in a second Just to go into one of these in a little bit more detail object detection, right? If we say this is a picture of the Eiffel Tower and you and I sat down and our job was how do we write a function to? Tell me if this is a picture of the Eiffel Tower We I mean we would quit right we there's no way we could actually write code to do that There's too many pictures of the Eiffel Tower on the internet We're in the world or possible configurations of ways you could take it and getting something that was highly performant It would be too hard for us to do as humans I Assert to you if you have a different opinion. I'd love to hear about it afterwards, but I don't think I don't think you're right So convince me otherwise and that's why we have this thing called deep learning so Machine learning sits at the top. You know what that is now under machine learning There's a technology called neural networks right neural networks are just a an application of a way to do computing That's inspired by how our brains work. It's not exactly the same, but it follows a similar model What's cool about neural networks is that a lot of really smart people in the field have proven that Given enough computational power given enough memory There is nothing you cannot compute with a neural network style right so anything all the apps we build Everything right on Matteo and Dario's parser We could do it with a neural network anything that you do with your day-to-day job We could do with the neural network if we had you know enough inputs and outputs and or you know computing power to handle it deep learning It often uses neural networks and the reason why is because in a neural network Basically, you have these layers of neurons and I'll explain what that is a little bit later and data flows through these Layers from an input layer which in the case of like an image would be all the pixels of the image to an output layer Which in the you know in case of an image classifier would be here's all the things I know how to classify this was a picture of the Eiffel Tower This was a picture of a cat right however many things I know about I'm gonna have a bunch of Numbers on the output layer one for each class and hopefully the one with the highest number Matches the what we actually send in and that's how we we decide what was in the image. I have pictures It'll make more sense when you see pictures But it's called deep learning because you have multiple layers in this thing it's not just input one layer output and it's this magic that kind of happens when you you stack a bunch of these layers together that Computers get really really good at this stuff That might sound intimidating, but the truth is you don't need a PhD to do this and you don't even need a ton of data We're gonna train an image classifier with very few images and this this blew my mind right before I got started learning about machine learning and deep learning in general. I have this misconception that you need Amazon size data sets in order to do this stuff. Well, you don't and and I'll prove it to you And the cool thing is you don't even need a dedicated team of data science experts to do this either Any of you who can you know write a bit of Python can can do this stuff today. I mean, I'm gonna show you how to do this So the use case all I'll give you is we have a service called Amazon recognition at 8 it's really cool You give it in it does a lot of things But one of the things it does is you give it an image and it'll tell you what it thinks is in the image, right? Like oh, I think there's a skateboarder here. I think it's a sport. There's a human in the in the picture. There's cars, whatever That's it's really cool. It knows how to recognize a lot of things But customers come up to me sometimes and they say I love recognition But I want to apply it to images that are specific for my domain that recognition doesn't know that which makes sense Right because we can't train a model AWS that knows everything in the world because we don't have all of the things You might have specific things in your business domain, right? Like I want to train it to recognize Gucci versus Prada handbags. I don't know, right? So it might know it's a handbag for example, but it might not know Gucci or Prada So if that's if that sounds interesting to you and you want to do this for your own domain What are you left with? Well, you can do it yourself. You can build your own image classifier That's what we're gonna do. We're gonna do it right now But first we need some data So now I'm gonna switch back to my computer and hopefully we'll get some some data captured Now if I talk to you like this for a minute without the mic because I need both hands. Can you all hear me? Okay, awesome Okay, sorry. I'm just waiting for the computer to decide that it's gonna show me screen good. Okay, I have two little web apps here Am I still running it? Yes, good All right, so I have a little web app here. It's really simple. It's got a webcam fired up It's gonna capture some pictures of me. I'm gonna call this My data set name pi sg So all this is doing is it's a web app It's gonna capture webcam images and put them inside like folders on Amazon s3 s3. Does anyone not know what s3 is? Okay, well further recording for folks at home s3 is a service that lets you stick as much data Basically as you want on the internet at unique keys. So it's like a data blob storage service And so I'll say human. I want to gather some pictures of human and it's just gonna be me here So let me just get some pictures of me here. All right, that's 25. Let me do 30 Okay, there's 30 pictures of a human 31. Let me get rid of one of those All right next let's just I don't know how many classes wouldn't do we can do at least two classes Let's do bottle. So now I'm gonna take some pictures of bottle There's 32. I want to keep it a nice even 30 for this Okay, there's 30 and then like maybe just for the sake of it We'll do one more and I'm just gonna call this one clutter So this will be things that I don't want, you know If it only knows about human and bottles anything I show it It's gonna be like oh, that's definitely human Even if it's not because it has to make a choice at the end of the day between the classes It knows about so I'll make one more I'll call it clutter and we'll just take some pictures of things that are neither humans nor bottles like I don't know Just the empty background right or like maybe some stuff of like I don't know these this this phone cord here my arm And then there may be this phone. That's not a bottle How about my hand? how about this soda can And I don't know that's probably enough. Let's see how it works Actually be nice that there were roughly the same number of images in each one. So I'm gonna do a few more. I'm gonna do this adapter We're gonna do my bag Okay Sorry Oh, we can do my hat too sure I will I'll get rid of some of these ones of this can And we'll just do just the hat All right Fun, that's probably enough. Okay. Cool. So all of these now live on Bucket is what we call the place where this data goes on s3 The next thing we're gonna do is we're gonna train this model now in the for the sake of time I'm not gonna show you how to spin up a Jupyter notebook in this tool that I'm using but I'll tell you about it later I Already have one spun up. I'm using a tool called stage maker I'll tell you more about stage maker in a minute when we get back to the slides But while we're waiting for Jupyter open, I'll show you that really what this is is it's a It's a virtual machine in the cloud that AWS hosts for us and it comes pre-installed with everything you probably want Installed for all of your machine learning and deep learning needs from a low level if you're like if you already know this stuff Do you want to just like write some keras or some tensor flow or use glue on or mx that or any of that stuff? We got it right you want sci-fi numpy all pandas. It's all there But it's also, you know, it also gives a hosted Jupyter notebook session, which is nice so Now I'm just gonna give it a few different At the points of different bucket and now I'm actually just gonna run all these cells This is the boring part where I'm just I told you I'm just gonna run a bunch of stuff And then we'll come back and we'll look at this code Okay Fingers crossed prayer to the demo gods. Hopefully this is gonna work and Mike I'm sorry that I just shouted at your levels, but hopefully it's okay Okay That step is done. So following a cooking metaphor What we've just done is we put something in the oven, right? So now let's I'm gonna teach you about the concepts a bit more then we'll actually read the recipe Well, then we'll look at what's in the oven. Oh, thank you Then we'll look at what's in the oven if it looks good Well, I do a taste test and hopefully we have something that works if it doesn't for some reason I have a pre-trained model. I did earlier that I can also show you but it's not a spawn. It's always nice to do live demos, right? So what is a neural network in more detail? Okay, so a Neuron You can think of a neuron as it's just a thing that holds a number. Okay. It's like a cell in some graph It could be point two could be pointed. It's usually a decimal think of them as you know like just decimal numbers It's not now imagine we wanted to teach a computer how to classify Handwritten digits. So this is a classic machine learning example So in a 28 by 28 pixel image of grayscale you have seven hundred eighty four numbers that you care about So for example, these numbers could be like low numbers like 30 if it's at the edge of the Drawing or you know like one if it's right in the middle. It's a really strong line Or maybe somewhere in between right in in machine learning speak we call this the neurons Activation value that's just jargon. You don't need to know what that means. Just think about a neuron Have is just a node that has a value Now we're going to take this image and we're going to like stick it all in a line right we're going to line up all the pixels And that becomes the first layer of some kind of network in this case This is a network that has two middle layers a week. Sometimes you call those the hidden layers And then at the final stage we have this output layer right now that the output layer has One node per class of what we're interested in classifying so in this case we want to classify the digits zero through nine so we're going to have ten cells ten nodes in our output layer and What's going to happen is all the numbers from this nine right that that became that this this first layer of input are going to go through this network and some simple math is going to happen and at the end of all that simple math there's going to be numbers here on this side and Hopefully the number that in this case for nine the number in the nine node there is going to be really high and the number for all The other ones is going to be really low and that's how the computers can decide. There's a nine there now How how does that happen right? Okay, well before I get to that let's just recap like what what is a neural network? Neurons are just nodes of the value. They're linked to these other nodes with various connection strengths so before you just saw lines, but those lines you could think of them as like being thick or being thin and those connection strengths can vary and based on the connection strengths and something else called a bias you'll get that's how the computation is going to Hopefully you know end up with being valuable for whatever our domain is The bias is just another value that that is going to determine if that node is excited or not right I like to think of this like like a joke right like you Imagine all these neurons are all lined up in a row and some of them whisper to other people other ones and then they whisper down the Chain and the the joke goes in one end and then some neurons have a sense of humor And I think it's funny and they're going to pass the joke on and some neurons think it's not so funny And they're dug in the side now. I don't I'm not I'm not passing. I'm not telling my friends this joke. It's not funny That's what the biases is like, you know, how how giggly are you essentially right? So in a simple example Imagine that we just had a two pixel image and we had a one node network This doesn't really doesn't make much sense But I wanted to illustrate the math in an explicit example So if we have values of and I'm using integers here because integer math is easier for me So you have a one and a five values going into this node with weights of two and three and a bias of negative four Means are going to do one times two plus five times three plus negative four and we get a value of 13, right? Literally that's like in its simplest form That's what's going on here. And it's just a lot of that chain together makes the magic happen I'm simplifying a little bit, but this is like at least 80% what you need to know in order to really just understand that It's not really magic. It's just a lot of math happening really really quickly really fast over and over again Okay, but how does the network learn right because what we could do is just keep trying random connection strengths of all the various, you know combinations Until we get one that works right this is like a million monkeys and a million typewriters and you wait infinitely long and you eventually get Hamlet Yes But that's not what we're going to do here because you know the sun would explode probably before we finish this computation There's a better way So in this example, let's say we're trying to feed a to through the network, right? Here's what's going to happen. The to goes through the network and you know The network starts out with some amount of weights and biases that just think of them like being random And then at the end of flowing through the first time the network is going to suck at it I have no idea what it's doing and you're going to end up with values like this, right? Just for example where some of the values are are going to be Really low and some are going to be really high But to change these the end results all we can do is adjust the weights and the biases So you can see here what we wanted to do was get the number in the two-slot really high Right, we know it needs to go up and it needs to go by a lot And we know that these other ones that are not to all need to go down now Some need to go up by a lot based on the current values some need to go up or get nudged a lot Some need to get nudged a little bit That nudging however, you know the factor of how much you need to nudge in what direction? Let's you With some clever algorithms that you don't need to write because they've already been done for you Basically start going backwards from each of these nodes and knowing how you want to change them and Adjusting backwards through the network that's called back propagation. That's just another jargon term And I've just described for you what's going on there. It's it's not really Complicated, I mean sure there's a math involved and you could figure it out if you needed to but again You don't because all these deep learning libraries already do that for you So let's recap we start by randomly initializing our Our weights and our biases once at the very beginning then we're going to send every training example through the network And we're going to measure the difference between what we wanted to see in terms of like the each output note and what we actually saw And then we're going to go backwards to the network adjusting the weights and biases And we're going to redo it and we're going to do this over and over and over again a lot a lot a lot a lot a Lot of times hopefully honing in on networks that perform better and better and better and at some point we'll stop because Either we've decided we want to stop for based on how much time we're training for or because of that Then the network stopped improving it got to a point where it's like how you could keep going, but I'm already good enough Finally we're going to run some validation examples through the network So this is different data than what we trained a network on and that's because you don't want the network to effectively Memorize your images only right you wanted to generalize Certain features about those images something I didn't explicitly say before that I think is worth touching on This is the cool power of of deep learning here in classic machine learning workloads There's a feature step called feature. I'm sorry. There's a step called feature engineering That is usually what you have a team of really smart data scientists for right you've got some data like credit card Fraud rates and then you have a bunch of transactions and then you know a team of data scientists goes away and they look at And they go We have 30 attributes about transactions and we figured out that these are the ones that are relevant for predicting If something's going to be fraudulent or not. So let's train a model on that Figuring that out and like how you need to tweak your data before you feed it to the machine learning algorithm That's called feature engineering deep learning gets a it does away with all that You don't need to do feature engineering because the network figures it out the features for you If that makes sense Now there's another thing specifically used in the field of image recognition I think is worth calling out here and that's something called a convolution jargon alert It's actually not complicated to understand But you might have heard this term convolutional neural network specifically in the field of image recognition a Convolution just means let's examine neighboring pixels together Okay, so in this example, we've got let's look at this image and let's look at all of the eight images or a pixel sorry pixels around it and Let's do some math on that right and so up here. We have a matrix It's basically just like a map right that says take the the pixel I'm looking at it the one in the middle there and you multiply it by eight take all the surrounding pixels and multiply them by negative One add all of that up and you get a new value And so this this called a kernel a convolutional kernel just think of it like a little map, right? It tells you what to do. It's the transformation This kernel Ends up giving you something that looks like an outline of an image Do you remember like the first time you ever saw like, you know in Photoshop? You could run those filters and you could just say like You know find me the outline of the image or you know sharpen this image and I used to think like wow How does it do that? It's actually really simple math like you could you could yourself any of us probably could sit down and Program a computer how to do that right? It's that's that's neat I never knew it was that simple, but it is so what's cool about this is with With some very simple math again The computer can figure out how to see the image in different ways that might be enlightening in order to help classify the image better I have a video of this in action, too So you can see more of what I mean So as we move the mouse around you can see that it's sampling the pixel in the middle Showing you all the the math involved for how it's getting how it's translating those images into new values On the right-hand side so for example that kernel gives you something like finds the outline of an image Sharpen you know that just means let's use five for the middle value negative one for like the north south east and west and ignore the diagonals And you get this like sharpened image if you were you can also do ones to figure out like the left edges of an image or the right Edges in the image etc. So the point here is we don't just look at the pixels one at a time We could do that But there's more information in the 2d context of an image specifically and so when you're building image Recognizers you want that This is the cool magic of convolutional neural networks. You layer these convolutional layers together So in each layer you could think of the layer being responsible for deciding on a particular kernel configuration And then you have several of those kernel configurations kind of pipelining transforming an image all the way through After one layer networks know about contrast and similarities after two layers They know about things like lines circles gradients after three layers. They know about things like Text is a thing Human shapes are a thing right? They don't know what it says, but they know like that that's a class of things that I see By five layers. They know I can recognize dog heads versus cat heads I can recognize bicycle wheels as a thing eyes of animals with centers of flowers. It gets really specific really quickly It feels like magic. It's not magic It's just a lot of math so How do we make our own custom image classifier in 21 lines of code? I'm gonna I'm gonna be a little bit I'm playing a little bit loosey-goosey with this 21 lines of code number You can look at my Jupyter notebook later and count for yourself the reason why is some of the lines I'm like, you know, I'm I'm being more expressive than I could be otherwise You could code golf this and even less lines in 21 and a lot of these lines are going to be data Configuration not actually doing training, but let's go look I'm gonna come back to this the if we have time if we don't have time I will just quickly say AWS has a lot of services covering all stacks of machine learning that you might want to run of the top layer We have services that are fully baked for you So if you don't want to train your own networks for anything, but you want to make life-like speech We have Polly you want to recognize images and video We have Amazon recognition image and video you want to take what I'm saying right now and turn it to text Even live streaming we have Amazon transcribe. We have comprehend etc. So you can go look at all those on The other total side of things is like I know exactly what I'm doing with deep learning Networks Gabe I already know this stuff. I want to use TensorFlow. I have this opinion. I want to use PyTorch I want to use Keras all that kind of stuff. You're a machine learning practitioner all the way through already We have Really great virtual machines with very very fast GPUs That can be elastically scaled to at your workloads. And so we serve you what I care about is actually this middle layer Right where we have SageMaker. So SageMaker is our service that makes it really easy to build train and deploy your models In a way that makes it Accessible for what I would say is probably most of us in the room, right? I'm not a deep learning expert I'm a programmer. I know how to write Python I know how to take data get it into the shape that some function wants So that's basically what SageMaker gives you so SageMaker gives you a few things But what we're going to show is it comes with a lot of built-in algorithms So you might not know all of the the different, you know algorithms for deep learning out there But you might have heard some of these things before XGBoost right or or K means clustering or random forest cuts etc. And if you don't our documentation will tell you it'll say things like If you have a problem that you know, you need to discreet answer look at these built-in things If you have a problem that doesn't know so there's good docs to help you You don't need to know ahead of time, right? I want to use this algorithm But one of the built-in Algorithms that SageMaker has is you want to make an image classifier? We got that for you All you need to do is give us some data So this is what we did in our Jupyter notebook and now let's go back and let's look at the notebook And we'll review the code a bit more. Oh And we'll see if it actually finished building it should have us. There was some there for some reason. Okay the moment of truth Good, this is good if we see lots of output here. That looks good. Okay, cool. I did something. Let's review the code I'm gonna make this bigger Is that big enough for everyone in the back? Hands up if it's good. Good. Thank you. Okay, so what's going on here first? We need to configure our training data, right? A lot of this stuff is gonna be about that So, you know, I am gonna pull up a chair because I'm gonna sit here for a few minutes Okay, we're gonna basically put our images in different directories one per class I hope that makes sense to everyone. That's what my tool did and automatically uploaded them to these like subfolders on our S3 object service So here we've just defined. Okay. Where what's the my bucket name? I'm not gonna win any awards on bucket names. Well, that's because I generated one randomly And what's my dataset name? We're like my subfolder in there where all the images went So inside PSG we have a folder for face. We have a folder for or for human rather We have one for bottle. We have one for clutter Next we're gonna do some things. We're gonna import our SageMaker Python Module here and we're gonna import a couple other things Roll and sash are just variables we're gonna use later on. I'm not even gonna explain what they are But you can like you can read the docs if you really care You'll you'll be able to infer from the context training image. Don't let this mislead you This doesn't mean an image like a you know with pixels This is a container image like a docker container and that's because behind the scenes SageMaker is powered by containers and so they the built-in image classifier algorithm that comes with SageMaker is just a Prebuilt docker container that you're pointing SageMaker at so point being if you have your own machine learning or deep learning Workflow already done and you just want to leverage the other things that SageMaker brings to the table like parallel training without you know Like automatic elastic scaling parallel training or model hosting and deployment But you've got your own you know algorithm written you just need to containerize it up and point SageMaker at that We've just done that work for you for a lot of common workloads So this training images gets me the image on our elastic container registry that the algorithm happens to be at for image classification Okay, so I'm gonna use a script here called m2wreck.py What this is is a script that will take my folders of images Convert them into different files for me And this is because I looked at the docs for the image classifier algorithm in SageMaker And it said you can pass me images directly on S3 or you can pass me something called a record IO format The option is yours and I said okay Well, I want to show people the harder thing because it's not actually hard to do And it's worth knowing how to do it in case you ever want to make a record IO files So a record IO file is basically just a one binary file that has all of your training data in it And it also has the class data that that says right these are my faces or my humans right these are my bottles, etc And it's a way that makes it very efficient for loading and distributing deep learning workloads like this So all of this ceremony I'm just gonna send some environment variables here and then all this is just saying okay I want to find this file in my path right where is where m2wreck.py exists inside Python's path And I'll store that as environment variable also Okay, now we're going to pull our training images from S3 This is me just using the AWS command line interface to say sync a local file directory with my bucket And now I'm going to make my my record IO files So to make a record IO file with m2wreck There's two steps the same script as both and I read the documentation right here And that's how I learned about it by the way this notebook at the end of my presentation I have a link to it so you can play around this yourself of course First up you make something called a list file a list file is just a file that says For every file that you're going to use for your training right this is the file name effectively This is the path on S3 plus what class it is So it's just a metadata file that describes all of the all of the files in what class each of those these images in In this case then the record IO file bundles that up into one big binary, right? So you use this tool twice to generate the two files and there's the the command line arguments are pretty self-explanatory And excuse me and the documentation is pretty good I should not have gone backwards, but luckily Jupiter notebooks keep all the output states that we can just keep going All right After that's done There's some cleanup with I don't need to clean up because it's already cleaned up and now we're ready to upload So this is me just doing more stuff So here I'm saying in this session variable that we did the top which gave me a default sage maker session This is a Bodo session if you've worked with S3 or sorry with AWS before using Python So this is that sesh was just a Bodo client session object And so from that it knows a bucket that it wants to work with for that sage maker has permissions to it to Read and write data from and to And so I'm going to say where I want my data to go in this bucket that I'm going to point sage maker at And I'm going to put it there. That's all this is doing. It's not very exciting. Okay Here's where it really gets interesting. This is where I'm going to say, all right. I'm basically making a data channels Object, right? It's just a dictionary that has training data and validation data So this is the fact that what I've neglected to mention before I'm sorry is that the into pie script I told it use 70% of my images for training and 30% for test And so it even takes care of splitting those for you I thought I was gonna have to write all that code, but I didn't so that's great That's one reason right there if you need to do this kind of stuff you want to use into records It'll it'll do that for you So now that we know what data is going to be for training and for test because we ended up with two Record IO files at the end of what we ran above now. It's time to train. So here's where it gets really fun So I'm using something called a sage maker estimator How did I know to write this code? So what I haven't showed you and this is a good quick segue Sage maker in the Jupiter notebooks that come with it when you spin up a notebook instance It comes with a ton of example notebooks for you to look at for all kinds of use cases, right? So let me just move this a little bit so you can see We've got introduction to machine learning in a bunch of different contexts But then we have all the built-in algorithms right introductions to all those that sage maker can do and even inside image classification There's like 10 different examples here that highlight different ways of doing things depending on what you're interested in right Do you want to use like 30 GPU or like you 30 nodes in parallel training? We have an example for that Do you want to do it with list file instead of record IO file? We have that Do you want to do transfer learning which is what I'm doing and I'll explain what that is in a minute We've got that so I actually started my notebook from this example And then I forked it to show a bit more because I wanted it to be a little bit more explicit than what the default notebook comes with so And then there's documentations for this to you can look at the Python sage maker SDK and it will explain You know what's inside this estimator class and what you can give it so all you give it is right What's the docker image? I'm building from that roll variable that we set at the top What's that? This is so cool this train instance count equals one if I have a lot of data. I don't have a lot of data I'm you I'm totally over killing this But if I had a whole whole whole lot of data and I wanted to train on more than one virtual machine in the cloud I just change this one. I could change it to 10. I could change it to 100 I could change it to 2,000 and it will parallelize my training You don't have to set up any of this infrastructure. Please don't spend time doing that That's why sage maker is so cool. I can just parallelize my training with literally just changing integer value here This next variable says what type of machine do I want to train on? I'm using our latest generation Virtual machines are p3 instances. I'm using the weakest one which is called a 2x large This machine costs $3.06 US per hour in the region. I'm running it in that's not very much money at all For especially for what you get so this machine has I believe You know don't quote me on this it has at least one Tesla V 100 GPU It might have more than one we can look at the specs later. It's a really beefy GPU I didn't explicitly mention this yet because I this is an assumed piece of knowledge, but I should say What if you don't know this about deep learning especially in machine learning in general? It requires a lot of math, right? Well GPUs are really good at math And so all the training that you're probably going to end up doing or you hear people talking about doing Requires lots of powerful GPUs. You don't want to buy that hardware. That's the advantage of the cloud You're only going to let AWS buy that hardware and maintain it and you only pay for it while you're using it And then you don't pay for it anymore Okay, and where should the model go after we're done right in this s3 out locations output location So it's going to get the data from s3 train a model put a model back on s3 We're gonna set some hyper parameters jargon alert hyper parameters just means config I really wish back when they were like coming up with the jargon here They just called it config, but I guess hyper parameters makes you sound smarter. So they went with that I guess they were thinking the data that goes into my machine learning training That's the parameters of my of my algorithm. So what about the configuration? Well, that's parameters about the parameters So we'll call it hyper parameters. It's just jargon So there's two types of hyper parameters that I think are worth just drawing a distinction between the ones that you don't want to Mess with in this case and the ones that you do. So what don't we want to mess with well We don't want to mess with these use pre-train model equals one this means Do transfer learning and I learned this by reading the documentation because I when I first got into this I didn't even know that stage makers image classifier algorithm does transfer learning. What is transfer learning? This is the magic that lets you train networks without a lot of data in this case They're taking a network that was trained already to classify many many tens of thousands of different types of things in the world And you take that same network that's already trained all the weights and biases for how to recognize dogs and cats and fish and you know People and whatever like tens of thousands things and all it does is It takes that knowledge and applies the same weights and biases to your data, right? So it's just solving for that last layer in the training now It doesn't have to mess with all those previous layers So you could also think of transfer learning as the feature engineering from the network was already done for you It knows all the relevant things of in this case about what we might see out in the real world And it's going to use this going to bring that knowledge to bear on the images for our domain What might we want to mess with? Well some magic numbers here like the learning rate and the mini batch size This is more jargon for you learning rate is just a number that represents How much should I be nudging my network by when I'm trying to do successive iterations? Mini batch size means how many images should I be feeding through in this case images? Should I be feeding through my network before I kind of look at it holistically and say? Did it learn did it learn well or not? How much should I know do my weights by rinse repeat? You can learn all this with more resources that I'm going to give you at the end of this talk So don't worry if it didn't make a whole lot of sense The point is these numbers are good to mess with and will give you a model that performs better or worse So you could be silly like I was when I first got into this and start Training your model over and over and over again trying to get better performance by manually tweaking these numbers But we have a why that's what computers are good for so SageMaker also has a feature called hyper parameter tuning or automatic model tuning Which you say these are the hyper parameters that I'm interested in you messing with SageMaker Go and train X times right where X could be any number you want like you could say go Go run 500 training sessions messing with these variables and then tell me which model perform the best Perfect now. I have the best train model and I know which which parameters gave it that the best model So if I want to play a little bit more around that space I could And so these are this kind of all the the the hyper parameters that go in and now we turn our model training It's just as easy as basically saying take that image classifier object I made and call fit on it pass it the data channels that said these are training data here to test data I give it a job name if you want I want to see my my logs And then these are variables here or just so I could see some stuff at the end and here's the logs now I'm not going to read all the logs to you because that's not really fun But what you can see here is there's some preamble stuff where it says, okay? I'm getting ready to go Here's all my parameters so I have that and then it starts training and what you can see here is the very beginning It trained and its first accuracy was only point four five like 45 percent accurate on the on the on the data for the training set Not very good, but then it also ran it through the validation after seeing it. So it was 88 percent accurate Okay, well and then it just rinses and repeats right it keeps going over and over and over again Each one of these things every time it runs through all of the images in this case five at a time because my mini batch Size was five every time it goes through all the images. That's called one epic So it does this over and over and over again, and as you can see very quickly It starts performing much better validation accuracy one one, you know hundred percent training accuracy point ninety one validation accuracy here Point nine six and you know it goes for a while and it starts honing in on something That's it's really really really good in this case It ends up being you know one one point oh for training and for validation at the end which is awesome So this is line. I want to draw your attention to 194 billable seconds So remember before I said this instance. I was training on was three dollars and six cents an hour But we only use a hundred and ninety four seconds of that right so what is that that's 306 cents per hour divided by 60 minutes an hour divided by 60 seconds in a minute times a hundred ninety four seconds is 16 and a half us cents to train our custom image classifier Come on guys. That's like that's such good value. You have no reason no excuse not to be playing with this stuff In my opinion because that's cheap And then now that I have a model trained. What do I do with it? Well, it really is it's just a It's a bunch of files inside a gzip file on s3 that if you unzip this you'll see it's there's like it's some Json it says here's what your network looks like here are the weights and the values and then basically Something is going to load that into a you know programmatic representation of that network so that it can feed data through it Hosting this yourself not hard you could do it but again you probably want something that's going to be highly available and Scalable and and secure and those are things that are not you need to your domain and you probably shouldn't be worrying about those So why not let SageMaker host the model for you and that's what it'll do So with one more line in this classifier dot deploy and I'll say I want one Virtual machine to host my endpoint to start with It will it'll spin that up for me now. What's important to note here is look at this I'm using a different instance class type here. This is a t2 medium. This is a very very cheap Relatively low powered CPU and low memory virtual machine in the cloud cost a lot less than three dollars an hour I'll tell you that we could go look but it's it's not very much money at all The point being training your models you want GPUs which cost a little bit more money Hosting your models oftentimes you don't even need a GPU on the box You can just do the inference on a CPU only you'll always get better performance if you use GPUs But you often don't need to depending on what you're doing. Okay. It's deployed now. It's time for the candy Let's test this out, right? Here is the endpoint name And now we plug this into my other tool. Thank you for bearing with me for this, okay? My other tool is logo host port 3001 and I'm gonna say My endpoint name is that and my classes were what clutter and Bottle and this needs to go in alphabetical order Bottle clutter and human, right? Okay Here we go Classify me. Hopefully this works. Good. It thinks I'm a human. Can you guys all see that number in the bottom there? And 0.94 percent certain that I'm a human there not bad for literally 30 images for each training set And you can see here it definitely didn't think it was a bottle or clutter those numbers are really low Let's just do another one now for another one I'd like a volunteer from the audience and I'd like somebody who looks substantially different from me and You'll you'll maybe you'll see why in a minute. So can I have somebody come up who doesn't look like me? I eat not a white guy Anyone please. Thank you. Come here. Just stick yourself in front of the webcam here and then click that classify button, please Yeah, I'm just yeah, click click there Okay, thank you All right. Now it thinks he's a human too, which is good. He looks human to me. Are you human? Good But as you can see it's a lot less confident, right? It says a point 82 percent confidence score or 82 percent rather that's not as high Quick little side note, right when you train your models, there is bias You know, it's very important that your models only as good as the data you put into it garbage in garbage out We already know this from all the code. We write it's the same for deep learning It's not so magic that it's going to know every key what every human looks like It knows some of the features that make up a human especially for white guys wearing hats Let's see how it does without my hat on See point 62 right because all of my training images. I had the hat on too So again, you want a variety of samples of data? I intentionally didn't collect that during the training set so I could show you that it doesn't do as well when you You know when you bias your data, but sorry is there a hand up? I'm sorry. Oh, yes So we'll get to that. I wanted to try the other ones too. Everybody always asked us to it's the best part But first let's just do one with the bottle so we can say how it does with bottle And we'll do a clutter one and see how it does with clutter and then we'll we'll try some other fun things Now that's a bottle that it was trained with right it saw that bottle before let's use an Evion bottle, which is different Let's try and like give it like I'm dripping water on my computer. That's not good Amazon won't like that Let's try it at a different angle to like a really sharp angle. Maybe down Okay, let's see how it did bottle point so 98 percent bottle 69 percent bottle 55 percent clutter So see it didn't do very good with the bottle head on again I didn't take any images with the bottle head on so that's not surprising So you want some variety of data, but it's still pretty amazing what I wanted to show was they can perform very well With transfer learning even without a lot of data Let's do one with a bottle over my face because I was requested Okay, I've given you a few different looks. There's bottle in middle of face. It thinks bottle There's bottle way in front of obscuring my face except my shoulders. It's definitely bottle 97 percent There's bottle covering half my face bottle 69 percent There's bottle to the side of my face so for whatever reason they decided if it sees a face and sees a bottle both clearly it's going to pick bottle and You can train classifiers to do multi-object detection. That's not what we did here, but you can do that, right? So it's not like that a kid you can't it can't do that you can It's available for the purpose of this demo. We were only teaching how to recognize like the main thing or one thing in the image If you were Let me just do one more with clutter. Hold on And let's give it clutter. It's never seen before also like this microphone pack Just to see clutter See I thought that was a bottle so again, but only 50% confident So the other thing I just want to mention before we move on you don't just want to say Okay, right because you'll see we get back the scores for every class It right and it's up to the consumer of that data to decide what to do with it You don't just want to blindly say okay take the highest score that must be what it is Depending on your case. You probably want some threshold right where if I'm not confident enough about it I'm not going to do something with her I'm going to have a human manually review it and then maybe I'll feed that data back in and train again So it's even better right you can do all these things really important to take to never depending on your use case if it's really important like you're doing Something that's really critical. You don't want to get wrong. You want to have a very high Threshold, so you're only going to consider you know, maybe point nine eight or point nine nine And if it's really really really really critical, you probably never want the machine to be the only thing that makes the decision, right? Like I don't want to have a robot that's going to shoot people ever But if you were making something like that you probably also want a human being to decide It's time to shoot your tear gas at the person Instead of the computer doing it all on some right because that's just always a good idea when you're messing with humans more questions Yep. Yeah Okay So the question is my training data set was quite balanced I had 30 images of a human 30 of a bottle and then a few of random types of clutter Have I tried something where it was maybe five images of a bottle and 30 images of a human and see how it did I haven't tried that I would encourage you to try it and report back to the group and see how it goes Then then I don't imagine the results are going to be any different when I do it up in front of you here So a good piece of advice try and have balanced data sets when you can because your models will have an easier time learning from that I have a bit more stuff. I want to cover and then we'll wrap up That's not my clicker. This is Okay So we tried it it worked. Thank you demo gods. I'm always happy when that works just want to wrap up by saying look I Work for AWS. It's important for me to tell you more machine learning workloads run on AWS than anywhere else in the cloud I don't have enough room on the slides anymore to show you the logos from all of our cool customers doing awesome things With machine learning workloads on AWS But we can talk about just a few of them take Expedia hotels.com for example, right what they realized was some images of hotel rooms are really good for getting people to book and some are not so good and And so they trained model using transfer learning the same concept that we did today to help them predict which Which types of images would would convert better? All right Here's another one Grammarly. I see ads for Grammarly all the time I think it's a pretty cool service if you're not familiar with it It is a service that will watch what you type and then suggest, you know how ways to make it more clear Grammarly uses TensorFlow, which is one of the popular deep learning training tool gets out there and they liked working with SageMaker on AWS because of the thing I told you before The the elastic distributed training rather, so you just say I don't want one node I want however many you need because you have a bigger data set and and you don't have to worry about any of that infrastructure yourself So Grammarly had a lot of positive things to say about that then also after running the The training the model using SageMaker as they mentioned you can use SageMaker to run the inference yourself But you can also take that model that I told you about before that gzip file you can download it you can put it on iOS you can put it on Android and you know we're on a local machine and Have a client library there because again It's just weights and numbers representing the shape of the end the shape of the network And so with a client library that knows how to parse that it can then just feed data through the same way that the endpoint Does there's no magic there? Okay, so where can I learn more if you want to learn more here's here's your link dump This is the slide to take a picture of if you wanted there's two you should also take a picture of my contact info So you can follow up with more questions afterwards, but this is the one you definitely want okay those cool animations I showed you before thereby a guy named Grant Sanderson. He's amazing These were used with his permission because I saw them and when I was learning the stuff And I said these are the best videos. I've seen on deep learning. Can I show them to more people? He said yes, please do so he goes by the handle three blue one brown. He has a series on YouTube called Understanding neural networks that's 60 minutes total watching time for for each part like together total So if you have an hour over the course of one or two lunch breaks watch this it's amazing There's no better explanation. You'll find in 60 minutes in my opinion If you want more than our fast.ai as an online course they cover a lot more stuff But the idea is it's about practical applications of machine learning even in like the first course They they have you basically training and building your own image classifiers, which is really neat total content I'm there around 14 hours If you want a book instead neural networks and deep learning by Michael Nielsen free book six chapters online Because into a lot of depth if you prefer reading instead of watching videos or doing courses Finally should like a stage maker for running this stuff as I mentioned before it does a lot There's also a free tier on a WS stage maker Sorry Amazon stage maker so you can get started for free if you've never used a WS before and you're interested in and playing around with it These demos both the two web apps I built and the Jupyter notebook that runs all this is all on github so you can look at those two Machine learnings for everyone don't let anybody tell you differently Don't feel like I can't do this stuff because I don't understand the math or the jargon or whatever That's BS you can spend a little bit of time reading the docs and looking at the bajillion Examples in stage maker and build something tomorrow or this weekend because you can and it's really fun So what will you build? I hope you will keep in touch and tell me. That's it. Thank you very much. I'm sorry. That was quite long Before you ask your question if you can just tell me your name and where you're from because I like to know who I'm talking to please I Sorry, do you work to work somewhere? Okay, cool. What's up separate? I would just like to ask So for those 30 images and things right is your code that actually doing data augmentation already Okay, so let me unpack that for the people who are familiar with what data augmentation is The the question by the way was I don't know if the recording picked it up My could you do you have the these mics going into audio? Do I need to repeat the questions? No good Okay, data augmentation is a concept of saying if you don't have a lot of data like my 30 images only per class More data is better than less data And you know if I take a picture of my face and we look at my face and I flip it and so it's like my face in the mirror It's still a face right or if I tilt it like five ten degrees one way or another or I you know I put some noise. I make it a little grainy like it was taken at night. It's still a face We all recognize that his faces we can apply these sorts of augmentations Programmatically to our data set to make even more data to train from I didn't do any data augmentation for this Answers now Water from Taiwan and where do you work? Where do you work? I'm working on top of and I have been like a month old. Okay, I have trouble around cool Welcome. Yeah, I just saw this event on meetup so I just buy in and I appreciate your explanation, but I searched a lot of like a little bit cores or Information on online about a new role and where it's all about image Recognize the image is there another like a Implication event category. Are there other are there other ways to do interesting things with neural networks besides image classifiers? It's basically like you're asking the short answer is yes Trying to think well, I mean everything I showed you at the beginning, right? So let two of the examples at the beginning were face detection and object recognition, right? Those are both you could think of as like being in the field of image classification, but self-driving cars, right? I don't know this for sure But I would be willing to bet and have somebody prove me wrong that your auto self-driving Cars that that are on the road today, right now autopilot modes are using deep learning Models under the hood today. Huh? No pun intended under the hood. That's funny That's not just about that because like there's other input that comes into the network too, right? There's LiDAR sensory input That's not it's not an image data, right? There's the how fast are you moving and you know what inputs should I be making to the car? For example in order to safely steer it in the direction I want to go there's more than one way to turn a corner and start and stop at the same place, right? But like you know how you're gonna if you're gonna take it fast or slow might also depend on What road conditions you think there is based on you know the rpms of the tires for a given velocity etc All that stuff could be input into the network words figuring out what's relevant and what's not The other one I gave you an example of what sentiment analysis, right? So you could apply machine learning or deep learning to sentiment analysis as well If you wanted to for like is this a positive tweets with these negative tweets, etc Another question. Yeah, I have two questions. Okay It's very simple. What's your name and where do you work? Oh, yeah, I have questions first. Hi, Len Yes So the first first part of the question is because image classification By and large is Well-defined contour So my first part of the question is about the fine contour and what you have to specify Like a human like an object blah blah blah so Take Eiffel Tower for instance if you go to The internet you you look for Eiffel Tower you get like 10,000 pictures And some pictures actually look like Eiffel Tower. They are not Eiffel Tower in Paris So they look alike the one in China Yeah, ten times bigger. Yeah, so talking about that they have bannies and all every other fake city in the world replica in Down to the last detail. So how do you then tell from which country? So this is about well-defined images. So the second part of question is about undefined So things that don't have a contour or like water Why I bring out this subject. That's actually a lot of classification of images in this Area typically one question is Currently there's a sign it is called environmental DNA And this is very important because From I'm sorry I'm just gonna interrupt you in the interest of time because I it sounds like there's a lot of relevant background here But to respect everyone else's time here Let me just answer the question as you've asked it without I don't need to know the background about why it's an interesting question I already think it's an interesting question And if you want to tell me more about the background afterwards tell me afterwards. Is that all right? Can I just answer your question? So the second part of question is about undefined. Yep That means things that don't have obvious contours. Yeah, I understand. So Like water, yeah, so you if you have 1,000 rivers and 20,000 Spots of a lake or an ocean blah blah blah. I don't know. Okay. I think you know what my question Thank you. Sure. So your first question was about fakes, right or replicas of an image And how do you tell which one is it? Which one is it? Is it the Eiffel Tower replica in China? Or is it the real Eiffel Tower in Paris? I have two questions about that. What is the real Eiffel Tower, right? Like they're both the Eiffel Tower Right, it looks like an Eiffel Tower. It's an Eiffel Tower, right? Duck typing in the real world if it looks like the Eiffel Tower and then then it is the Eiffel Tower From an image classification perspective if you wanted to know specifically which Eiffel Tower it was You would probably need to have a more rich data set like perhaps the GPS data from the exit file on the image Exactly. I think it's obvious too Your second question was about Can these sorts of techniques classify Images without well-defined contours well like lots of bodies of water for example, right rivers and lakes and stuff I don't know. Will you try this yourself and let me know? I'd be really interested to know how it works out Okay, we don't have any time for any impressions. So Thank you very much all for coming and thank you again once again for that. Thank you so much for the opportunity I really appreciate it