 Ie ddeg. Fyddwn ni'n waith i fy Skylfe likio. Rwy'n fawr. Rwy'n fawr. Rwy'n fawr, rwy'n fawr, am gyfaniaeth i'r sgweithio cyfnodol. And so, hopefully you will cover both of the kind of deep tech piece which will really be an advert for the TensorFlow Meetup group, and all the way to AutoML. So, thank you for the introduction. I'm Martin Andrews. I've got a kind of background in machine learning startups for Nantes. from New York to Singapore in 2013. 2014 I lived in Singapore under the radar doing machine learning, deep learning, NLP for fun, and robots and drones. I converted myself from a finance guy to AI guy. I proved that in 2015 when I got serious working for a local company in Singapore and we've been doing natural language products there. Along the way I've become this GDEML, which is great, I guess, free labour for Google. I do the TensorFlow and Deep Learning Meetup here. I've written a couple of papers as a gentleman scientist and I've been running a developer course with my co-organiser for the Meetup Group. And with him we have also built a little company, so this is Red Dragon AI. We're a Google partner for doing kind of deep learning consulting prototypes. We partnered with SG Innovate to do educational training and we're also very interested in this kind of conversational AI, natural language voices or natural voices and knowledge bases. So that's about me. So I've done the Who Am I. I'm going to talk a bit about the TensorFlow ecosystem and how it's evolved. I can do a little demo for TensorFlow with Keras. I can talk about AutoML and then wrap up. But just so I know who you are now, so I've done the Who Am I, how many people have done machine learning anything before? OK, so how many people are at a company where they wish they had done machine learning of some kind? You wish you could be machine learners? No one cares. OK, I might as well go home. OK, so in that case you must have either come for pizza or dancing. How many people came for the pizza? Ah, so the rest of you must be, this is a kind of a new example from the deep learning field for dancing. So this guy, this guy can dance. These people cannot dance except if you use the power of deep learning. This is kind of interesting. So whereas the task from machine learning is to say good dancer, originally bad dancers, this is where the kind of things that people are now beginning to be able to do with deep learning techniques. Come on, they're going to do ballet soon. OK, bad dancing, better. It's surprising how limber the guy on the right has got. OK, so let me go on with the TensorFlow stuff now. So how many people have seen this picture before? This is from last year. OK, very few people willing to put up their hands. So basically this is how TensorFlow is a diagram which has grown from when TensorFlow was originally launched. On top of the TensorFlow, which is basically a means to get models built into something which you can execute. Basically, this can be accessed via C++ Java or Go, but the primary thing is a Python front end. And on top of that, there's various other libraries of layers of different operations. Keras is kind of an important way of doing machine learning as a high level API. And also Google have been pressing on to make estimators to make it so people can use like a psychic learn kind of interface. But also how can you deploy it? You can deploy it onto your CPU, onto your GPU, on Android and iOS devices. And also using this thing called XLA, which is like a compiler for this stuff. So basically this is grown and grown and people have kind of thought that they would make this diagram last forever. But from what I saw from when I put this together last night, people have given up with this diagram because too much has been happening. So one of the things which has been happening in particular to this middle layer, this TensorFlow main layer, is there are now more options for that layer. So you've got TensorFlow, which is kind of the original version. We've also got TensorFlow Lite, which can be deployed on mobile devices or as part of these cloud functions or whatever. So this is kind of a stripped down, efficient version which doesn't have all the bells and whistles. There's also TensorFlow.js. Basically if you can train a model in your big GPU setup and then deploy that into any browser. So this is a model which should execute on people's browser and give them machine learning functionality. You can do real time video overlays and all sorts of crazy stuff using JavaScript just in their browser. So they're a while times ahead for that because the JavaScript people is kind of a different animal from the Python guy. So that's kind of exciting. There's also a thing called eager mode which is coming along and the TensorFlow teams are excited about eager mode and maybe I should talk a little about why this is interesting. So the way in which TensorFlow works is you kind of describe your model to the TensorFlow and then TensorFlow goes away and crunches it up in some way and then it executes it. This is very different from if you're just writing a standard program where you tell it the steps you want to do. If you tell it the steps you want to do, you get an error as soon as you tell it the wrong thing. Here you would take your whole recipe for a real TensorFlow, you would take your whole recipe, compile it and then from the compile code you would say you made an error and the error messages are diabolical. Because you don't actually, there's no actual necessary correspondence between what it's giving you an error message about and the actual line of code you told it because it may be hidden away in the wrapping up. One thing which has come along last year is something called PyTorch which is another framework for building neural network models. And that works very much like a Numpy, like a standard Python library but using your GPU. And what has got Google concerned is that a lot of researchers are using PyTorch to build papers with. Partly because it's so much easier to debug if you make an error it just tells you straight away. So this is where they've now built eager mode which is a new mode for TensorFlow where instead of taking your graph doing all these operations and then finally letting you know at the end of the day how it's working, it tells you immediately. And so this is something which is suddenly as the talk about eager mode has ramped up and this is essentially PyTorch, not that Google would want me to say it, but it's kind of PyTorch mode for TensorFlow and it makes TensorFlow kind of very nice to use but it's got this huge ecosystem behind it of all this other nice stuff which PyTorch was never really designed for. PyTorch was much more of a prototypy kind of thing. So another thing that has just come along which doesn't really fit in the diagram too well is ingesting data using tf.datas. This is APIs where you tell it where my data is, it's a CSV file, it's all these images, you ingest it, so you pull it off the disk or whatever, you fill it around and then you can train models using it. And what people would normally do before is you'd have a loop in Python pulling this stuff little by little into the GPU, but the tf.datas that they are arranging it so that it can be all handled within this TensorFlow graph. And the benefit of doing that is you can organise it so that it works in an asynchronous manner so it's much more efficient, your CPU being kept busy while the GPU is doing the training work. But the bigger benefit, which Google didn't really, hasn't been so upfront about, is the whole thing with TPUs. Now TPUs are extremely fast pieces of silicon that Google has made because Nvidia licensing, Nvidia licensing, all these other situations, and also Nvidia chips, the GPUs which people have been using, have also got a lot of silicon devoted to graphics. For the machine learners we don't care about the graphics bit, we care about matrix operations. So TPUs, basically Google has rolled their own silicon and they make it into massive arrays of these things. These are pods, so this is a pod of TPUs, but it's a TPU is not a card you can put in a machine. In particular it's not a card you're going to put in a machine even next to your virtual machine at Google, in the Google Cloud. The Google Cloud virtual machines are in one place, the pods are in another place. So the benefit of having this TF data ingestion going on in the graph is that they can then transmit the graph into the pod and have it executed there. So your actual VM, if you've got a TPU attached, can be a tiny VM. As long as you can transmit what you need to do across the network. So once you've got this network over there, what about the disk? Where's the data come from? Wouldn't it be better if I just had the data locally? But the thing which amazed me is even if you have it on a normal local machine you'd have an SSD with all your data on to pump it into your GPU and everything will be happy. The problem with TPUs is that they're so fast SSD will not keep up with them so that you'll be starving your TPU for data even with local SSDs. But the neat thing in a Google data centre is that if you store your data in buckets, basically the data will be streaming into the TPU pod from lots of different directions simultaneously and the actual network transmission speed of this will overall be higher than SSD just because you've got multiple parallel routes into your pod. And so this is kind of crazy, it's a crazy thing where the kind of the old idea of having network accessible storage is a you know a slow problem. Actually Google has levered this with their data centres. I digress. Making it easier, okay, here's a key point. So Google has made also making this machine learning thing easier first by these cloud vision APIs. Basically these are pre-built models where you can ask okay what kind of dog is this or what kind what of these hundred thousand classes is this from or you know what landmarks are these. But there's also this new product which is kind of heavily pushed at the cloud next which was this AutoML. Basically this allows you to build on an automatic or allows Google to build automatically from your data um good models. So I'm going to talk about that a little bit. So let's go back to the kind of the raw stuff the the real deal um in the cloud. Another thing which Google has come along with is this Google collab. How many people have heard of this thing? Some people? Okay this is a super thing which people should everywhere should be using. If you use Jupiter and you want to do machine learning basically it allows you to use a free GPU from Google um it's a preemptible one in that they could be taken away from you at any time and the machine could die died any time and it's it will definitely die after 12 hours but you can then just have another one back okay. But it allows you to train on a GPU and you can also then save the models off to your drive or off to a bucket or wherever. So it is basically at work we've been keeping these going continuously for several months right. Just you know you you leave a job running overnight come back the next day it's all on your drive the machine died but it doesn't really matter you can then revive it continue training. So you get a free a free k80 um Google very nice of them. But the the reason they've got all these these k80s lying about is because the TPUs have taken over all the workload that these old GPUs did so now they've got these for free. They also built some kind this is for dollars now so Google's built deep learning VM images so these are all pre-installed with the latest Nvidia drivers tensor flows compiled to use this even better than normal um they also have the same thing for PyTorch which is a very interesting move so um hats off to them. Okay so now I will do a quick demo of TensorFlow and Keras but since no one here cares about machine learning I'm not sure why I'm going to do that but one thing says that this is part of a repo which I have so I've been running this kind of deep learning workshops for quite a while we've been running these um I have a repo with tons of examples in um but also now we're just converting over to running these things in colab essentially last year we ran a developer course where half the class had a GPU and half the class had a really tough time okay now everyone can have a GPU it may not be a great GPU um but just running this directly on colab from github um makes for a great experience so this is this is my Jupy this is a Jupyter network running directly on here because colab is not trustable in front of 200 people so okay so what I want the the the kind of the use case here and this is kind of a common use case that people will have is they've got a bunch of images that they want to classify and so this is a classification problem I would love to be able to show you the everyone can dance but this is not that's not for today so um and the way in which people do this typically is instead of building a network from scratch and training it on just your images they will take a prebuilt network and then kind of switch out what it's looking for so I'll go through this so so within a thing called Keras which is a higher level API on top of TensorFlow there is a model zoo and basically there are a whole bunch of models where if this is the number of parameters these these are really big models these are small models and this is the kind of the precision to be expected then you can kind of have this efficient frontier of of smaller and better models will be up in this corner and so all of these are available these all come from different waves of research as time has progressed so I'm going to take a model called nasnet which has a particular picture here so hopefully it's just going to load the model load the model of course it may decide not to load oh if it's not going to load it's a problem so the issue with them with an external model zoo is I have to call the model zoo come on okay I will pragmatically redo this okay I'm not sure I can trust this to work at all okay I will have to plug ahead let me talk a little bit then about this nasnet thing so rather I could show you this is this is a demo essentially I could I could show you online so it is available online so let me just talk a little bit about this nasnet and the reason that's nasnet kind of thing is interesting is because this is the basis on which some of this auto ml stuff is done so in previously what has happened with with all these architectures is that somebody or particularly like these big and bad ones originally somebody built these by hand using kind of old technology and there's a very laborious process simly that it then went on to inception you know reception one two and three and then there's these resnets all of this being built by graduate students essentially and so this is known as graduate student descent in that they would build good models and basically you'd use human labour to build these things and there's a limit to that and a limit to the scalability of that so what google decided to do was instead of using people to do this they would make people build models that build models so this is where this this is the net nasnet stands for network architecture search so basically they made the network architecture search into a game and so the game is played by coming up with a network architecture seeing what I can score an image net then then coming back and trying again but because I know what my essentially I come up with a structure I come up with a score I can use that as kind of a playing go I can use that as a do to five kind of thing I can learn how to build these network architectures so what then happens is basically it then builds basically the game has structured to build me a cell from which I can build an image net recognizer so this is the cell which has been built by a machine learning to build things and so this is where essentially this it will now become impossible for humans to compete in this game in that because the machines do this so efficiently and can be told all sorts of constraints that I want this as small as possible I want this to fill fit in this degree of latency extremely difficult for humans to play that game but computers can do it ultra well so this this whole thing has got more and more efficient these nasnet models on the other hand have just been released so you can download one of these things when the network through and just get these nasnet mobile weights directly and then use it so basically I'll get a single prediction from this thing I here's an image I read the image I show it and then I get a single prediction from this nasnet model and so here is a this is an image net prediction demo it's just book so um so this is coming back this is the image net is a standard image competition which has recently been abandoned just because machines have surpassed human accuracy and so now we essentially need something more more difficult to do um so this is it is basically a thousand categories of things including it clearly tabby tiger cat Egyptian cat Persian cat and links okay and this is pitch of tabby cat is fairly or fairly sure of that and basically I've got a a directory full of images here and so here's another one it thinks this one is Simon's cat not a bad idea this one I'm not a I'm not sure what it is anyway standard poodle maybe it image net the thousand to us is don't cover um like snowy white owl okay so it has no idea what this is so I mean sometimes it would say like golf ball or in any way that it just does not know right so it's but it would be relatively reasonably certain about the wrong answer okay and then this this one dingo could be um or maybe that may be in anyway so image net also has a very large number of dogs that large number of dog breeds in there for for no apparent reason um but these are also very difficult to for humans to tell apart so part of the way in which image net beats humans is by being good at dogs which is not anyways not is but it may make horrible errors because it has the image net has very few humans in there image net is not not good at human distinctions at all it may have person as being a class right um but it doesn't have it also has gorilla as a class and that's been a problem for google so um so there's a question of data and dignity as well okay so basically here here is the model which is loaded um this is a neural network with many many layers of these things so this is a very very long network basically designed by computer rather than human so what I'm going to do now is I'm going to take this model and do a trick so this is the the if I can I view the head up so basically what I'm going to do is I'm going to take this is the image net network and I can take and this is my input image and then I do some kind of black box mystery work with this nasnet and at the end I'll get back like probabilities for each of the thousand classes or not even probabilities something before probabilities okay now that I there would then do an operation called a softmax which would then give me a kind of probabilities and the biggest one is my choice okay so the biggest output in this layer is no tabicat or whatever it is but it will kind of have like answers for all of the other classes they'll just be very small answers hopefully what I want to do with this this repurpose network though is I'm going to strip off this last answer because I want to use it on my own classes I don't want to use the image net classes it may be that I want to classify for instance cars okay so what I'm going to do is take off this last piece and instead use these kind of pre outputs essentially basically this is a bunch of apart from this one all of the rest of it is kind of degrees of mistake so I'm basically I'm taking the the way in which it's made mistakes and trying to use that to classify cars or whatever to my own thing I'm going to use a model called an SVM it's very standard kind of your boss would understand this model thing okay so one of the nice reasons for wanting to do this transfer learning this is what this is called like this is that you can say well this dense black box complex cnn I didn't train you myself Google trained it using all of their resources this is a Google standard model okay and SVM brackets boss you've seen this before you understand SVM just fine right so in combination of these two it's got some industry tested this this this um here's a model which works and in in some ways that's much better than saying well I built this from scratch and I think it's pretty good right so um so this is this is the game we're going to play basically I'll I'll pull in some functions I've got these separated so these are cars I've got these separated into two directories one of which is called classic cars and the island calling modern cars and then I'm going to walk through this directory and then essentially convert this into a model image and then get these kind of these things called logits which is from this layer and I'm just going to build a like a list of all of these so I'm going to take logits for all these things so basically here's here's my training set and it's kind of a small training set I've got 10 images of each of these kind of cars right so here's here's some classic cars like these nice cars and then here's some modern cars so these are Exotica and so that is my training set now normally uh one of these classifiers require hundreds of thousands of images right if you're going to build this using from the ground up but because I've Google has already built the image net thing using hundreds of millions of images it's pretty good at images now all I need to do is figure out the difference between modern and classic cars so basically I can here is I'm taking the outputs which I've kind of stored for all of these and I'm building just an SVM over these these features and targets so this builds in this is a half second at most right so I can then go through this and and actually then find out the logic layer for new images which has never seen before and then use the SVM to classify those so here it here it say well this is a modern one this one is not a modern car this is a classic car modern one another modern one it thinks this is classic which is ridiculous it was a Prius another classic one so okay so basically you can see that we're getting some kind of accuracy which is surprising given that it doesn't know about modern classic cars apart from what it knows about dogs and flowers and other stuff right and so what one of the things here is that it's probably not doing what you think it's doing in as much as if I'm looking if I'm looking at the difference between this is a kind of caveat for machine learning in general if I'm looking at these cars and this you know kind of sleek lines and modern cuts and all this kind of thing whereas this has got kind of bulbous I don't know there are different things which I would associate with classic cars versus modern cars but it may well be that the what it's interested in is the shape of the wheels because the classic cars have ones which are much more like plates um which he knows about and he knows about plates of food right and the modern ones are much more like lotus flowers right so the actual lotus seed pods okay so the actual things which it's looking out for in these images may have nothing to do with cars at all or it may be that the modern cars are always parked on a certain kind of parking lot whereas the classic cars are in always in a different kind of context so you've got to be very careful about what it's looking for when you train these models but I digress so so this is basically so this is just 10 10 copies of each image produces a model which kind of works but you have to kind of also know what you're doing I mean you hopefully you can find a blog post which does this repurpose it jazz it up for your own good so that's that's kind of the that's and this is the kind of thing which we'll talk about at the TensorFlow Meetup and more so um but let's talk about AutoML so supposing you're not going to get into Keras and do that whole learning curve so this is for it's in kind of a new google thing they're super proud of it this is for people that have data they have money but don't want to invest in people sorry they don't have any specialists they want AI results okay because they they have an AI company and that's what they need so what AutoML will do for you is it will build you a usable model tuned to your data basically by playing that kind of nasonet game that I described before that they essentially use they figure out which of their models we most applicable to your kind of data and then let it cook right and once it's done that fully automatically you just add money because it can both train better and serve the model for you straight away so this is a picture for people who are not resources but more like overhead i management so basically you get your photo data set you put it through this magic thing then you get a rest api and they can also do this for um there's some translation stuff there's some natural language processing stuff and surely these kind of suites of models will come along and get better so i've got an interest of time or and do the demo of this it's pretty much what you see here so basically there's a data front end where you can drag and drop your folder of stuff into here and give it labels there's the train interface where you basically you pick your budget and then you press the train button okay now your budget includes one hour of free training but what it will say is it will do it's one hour of free training it'll tell you how many more hours it thinks it needs which you can get a pretty decent model after one hour of free um but it may ask for another 23 hours of 20 bucks an hour so you know that's that's for your boss inside maybe and there's also a nice evaluation interface so basically you can look at the your prediction accuracy all sorts of metrics how the training went and there's also a thing where you can say well here's a it's called a confusion matrix if i said it was a a rose or whatever you know it did it make a mistake what what are the classes is it getting confused by and having done that oh it's way too small okay um basically here's some python code you can just call this model and it will serve it to you and not just serve it to you give you back the answers same with um you can you know you can do this you can call it fire any rest api and so my guess is this this is what's definitely easy to interface and because it's google it will probably scale so this model will be usable repeatedly from any device and it was built fairly painlessly so if there's time i'm not sure there's time i don't want to i don't want to crush other speakers um but this is basically i all i would be showing you is i have another image and i can just show you that this model really works we sam trained one in new yw'r gwrs we can share the model i can put an image in it just works um but there's no time to to give you to show you the hour training process because that's ultimately really boring because all it is is google doing something in the data center so the advantages for this is it's kind of easy to use in theory um it may not be quite so easy to do the data ingestion because it's really kind of finicky about how it wants it how it wants you to upload these things you don't need to know any you don't you hardly need to know anything about machine learning you need to know about training and test sets and and what accuracy might mean and all this kind of thing and whether the model is any good um but basically everything's done for you they're training valuation metrics and you get an api immediately um disadvantages is not it only does classification at the moment um so you will not be able to make you dance right so this is this is an advantage of using red dragon ai is that we can make you dance whereas this will only make classified not good dancer we also get no no control over what's happening um you can't choose various splits you can't do this kind of things where um you might want to have control over this and the results if you've had no control you may also not understand what's happened um and you can't really poke around because it's kind of it presents you a finished product and it's 20 an hour in beta at the moment not sure how that goes up or down um clearly they want people to experiment with this um so conclusions i guess so if you need a model but you don't know TensorFlow this is a fantastic product and i think what google has done is they've looked at the number of people on earth or the number of businesses on earth and the realistic number of people they can educate and seen that this is a massive gap so better to get this in the hands of people just the the way to democratize it isn't to make it so that everyone can use it is to make or make everyone understand it for at least they could use it um even if you do know TensorFlow on the other hand it's good for a baseline um it you know you can put in your dataset to this it will come back with something it might be it might be really good on the other hand it might be a great baseline to show that you can beat google so not nice isn't it um but it probably wouldn't win a cattle competition okay if you've got experts building these things you'll probably be able to build it beat it just because the variety of models it will test will not be and the features it will understand will not be as good as um like specialists who really understand your business and if you start retraining this just because there's a habit it might get expensive so there's a wrap up okay so the TensorFlow ecosystem is exploding it's gone kind of way beyond its early stages and now where it kind of is probably too much is the problem so so there's a an embarrassment of choice right now um on the other hand where they said where they started out saying we would get to all these compiled models and all this good stuff it actually works now it will actually go on to your your phone it's available in the cloud it will scale it will do all of these things um and this AutoML is a allows for a much broader adoption of AI um but really it's just machine learning um the AI thing that's way down the road right now um so the stuff I have the the thing which took a while to load is all in repo on github if you like it please add a star that's my kind of kpi for this um and I'll do a few a few little ads in the two minutes I'm ready so we have this TensorFlow and deep learning Singapore meet up um the next one's on the fourth of September we'll be talking about explainability where if you've got a model is it doing what you think it's doing um there are a whole variety of techniques but people are kind of worried that this is um that's one of the issues with these black box methods right um all these slides are I'm not sure whether there's a all these slides will be available somehow I guess um what we try and do is we try and have a talk for people starting out talk from the bleeding edge like papers and some lightning talks if we can just anyone who's enthusiastic about what they're doing um I said this before we've run these deep learning developer courses of which I think all we're calling jump start is module number one um we ran this last year um it was gruelling I think for the participants but they got a lot out of it including good jobs I believe so um in each module we have kind of instruction and people have to do projects for themselves there's SG Singapore government funding for those people who qualify which has been citizens and PRs um and we really want that to happen because otherwise we'd be teaching a room full of foreigners um so want that to that's been actually one of the slow pieces in getting this rolling just to get all of these approvals through um we're using SG innovate to help us do that now because they're plugged in far better than someone who looks like me will be so that's good um in terms of the jump start thing this is a two two day course um the next one is on the 12th 13th of September um there if you go in through SG innovate down here for some reason it's under their talent talent developer talent development link um this is 600 sing um but there's also if you can understand it there's also discounts for for Singaporean so um but this is another two-day thing you get to play with real models from the very beginning um then there's one last thing is the the red dragon ai intern hunt so if you want the if you're in a position where it might be interesting to you and you want to do deep learning all day um we're very interesting people who have an intern period to cope with and I I know because I've done this with several people over the summer it's worked out super well they've come out with cool projects cool resume things and we get kind of people to plug away at these models which is kind of still necessary so um there we go thank you