 Hello everyone I how you doing? Okay, great awesome. I hope you had a great pie day yesterday and today we have a quite interesting day still ahead of us promising eye-track And I would just like to do my part and talk a little bit about the machine learning landscape that we have today So as announced, my name is Ico. I work for a company called ThoughtWorks and And we are a consultancy which means we build individual software for different clients and in different domains using different Technologies and we try to follow what we would consider the best practices. So we try to be agile actually agile Try to follow things like test-driven development continuous integration continuous delivery and and pair programming and we are also a huge fan of open-source software and So there's this one statistic that I found so this is where the arrow. This is the company and apparently we are Number eight on that list or whatever. I mean the statistic being used here is the employees the number of employees Contributing to open-source software on git. I'm sure there are other statistics But I think it gives a gives a nice idea that there is that we do have some interest and for a company Around 5,000 people. I think that's quite impressive. Yeah, especially compared to some other lower on the list who are Significantly work power All right, we also have an office in Singapore so check it out. We do some meet-ups I think Martin Fowler is coming next month and Obviously as everyone in tech we hire as well Yeah, so working for a consultant he kind of makes me a consultant, but I would consider myself a developer and The my my passion is machine learning and it started when I was at university. I wrote my thesis about convolutional networks on Mobile devices and then later continued when I was working at clients at ThoughtWorks and one there's one particular client that I would like to tell you about that is That was the online retailer in Germany the the second largest behind Amazon and What we did there was we use neural networks to predict Orders Based on browser session data, so we looked at the browser session data of users that were coming in and then predicted how likely it is that That user will buy something within that session and if it would be unlikely we would Give offer some discounts or something like like that but another about me we're here to talk about machine learning and So as as heard before multiple times and as we all know AI machine learning all these terms are At least let's let's not say overhyped let's say popular nowadays And I think it would be good to get a common understanding of of what these term mean at least in my understanding and Artificial intelligence and I mean we hear AI everywhere Especially in the AI track Artificial intelligence, I I would consider artificial intelligence I like this definition that the box for dictionaries.com offers a lot And this is by the way also the the definition you find when you ask Google to define Artificial intelligence and it it basically is The theory and development of computer systems able to perform tasks normally requiring human intelligence So that doesn't sound too crazy does it and and one example that I that I like is is a chess program where if you think of a person playing chess you would you would assume that person has like some level of Intelligence to be able to play chess Now if you make a computer play chess Then that is the artificial intelligence But if you know how chess works There are certain moves you can make and then in the next round you can make certain certain other moves You could basically brute-force that right you could rate like write a couple of if-else Branches and then you can brute-force all the possible solutions and then find the one that gives That makes you win That that chess game But then again brute-forcing and diverse isn't isn't that crazy, right? so for me and for also dictionaries the the definition of AI is less a technical more like a Physiophic, you know what I mean Yeah, rather technical but but more physiologically shit, right? Yes, who was it? Thank you very much Yeah, but then so nothing nothing fancy so far but where the fancy part starts for me is machine learning where This defining your algorithm and that then follows step-by-step to achieve something isn't really the case anymore like in machine learning where The way of solving a problem the way the algorithm works is actually defined by the data we put into a machine And if you put in different data or we put in more data the way of of How the algorithm solves the problem changes And that's where I that's where I think that the fancy part starts and then there are neural networks, which is the subset of machine learning Modeled after the human brain where you have like Neurons and they are connected to each other and send signals and then deep learning which is probably the most Hyped one is basically just a neural network with lots of layers It's like a deep structure And then there's data science and we've Heard that term before as well And data science is basically extracting valuable information out of or insights out of data and it can use these methods, but it doesn't particularly have to So the question is why do we need machine learning? So there are there are a couple of use cases where Machine learning just simply outperforms other approaches and one thing that machine learning can do is it can find Non-obvious correlations in a set of data going back to the to the online retailer there was was working with We had like 700 data points of browser session data like finding manually finding correlations between 700 points and someone buys something would be infeasible where throwing the whole data at a neural network Was rather successful Right, so as I said my passion is machine learning and last year early last year I want to do something cool and machine learning and I Remember this this quote of by by Larry Allison who was the the founders and CEO of Oracle Apart from it being slightly sexist I think it still holds true and let me maybe prove that by asking a question Please raise your hand if you use a patch key a patchy calf guy at work Well, there's I expected more actually So I did this at a different conference and like 40 40% raised their arms and the thing is the follow-up question is How many people of you think a patchy Kafka is the perfect solution to the problem you're solving and then half the arms went down I'm not sure what whatever. Yeah, I see not nodding. So I guess I guess it still holds true even nowadays that in The computer industry we just go for the cool new kid on the block so I realized that and then I realized looking at the landscape of of Machine learning that's out there. There's so many things and it's so overwhelming that I need to do my own analysis of What tools are out there and what they're good for and this is basically the what I try to do with this talk is lift The fork around the machine learning landscape a little bit I'll be focusing on Machine learning and then later on its subset of deep learning So right when you try to analyze the landscape of something that can be used to Analyze data I mean what what possible approach could you do? Obviously you do that data driven So what I did was I compared a lot of tools based on certain features I looked at the the popularity of these tools where I went to get up I mean I wrote a Python script to get star the star count of github for each of these tools and The same thing with the questions that were asked With this specific tag of one of those tools on on stagover flow. I looked at some API dogs I played around with some tutorials and when possible included expert opinion and all that to to get a better understanding of What kind of tools are out there and what are they? capable of That means If one of you have worked with I don't know Pytorch for the last two and a half years. You're probably more an expert in Pytorch than I am I Have used some of these tools, but definitely not all of them and Right, but the thing is they are lots of them are right new rather new like two three Maybe three and a half years old I don't think anyone in this on this planet can be an expert in all of them so that saves me a little bit in that regard So your question now is I could you spend all that time analyzing that landscape So what is the best tool and I have to disappoint you a little bit because I have to give you the classical consultant answer It depends It depends on a couple of things that depends on The use case you're you're doing What do you do image classification be it for signature checking or for optical character recognition? Or you want to do order prediction like we did at the client in Germany that I mentioned before and I think in general a good advice would be if The tool you're picking doesn't solve the problem you're having Don't pick that tool That might sound obvious I've seen cases where that advice wasn't followed It also depends on your experience like a team's experience regarding the programming language that is used regarding the tool themselves Also regarding the the general general understanding of machine learning depends on the infrastructure whether Your model will be running on a powerful server or on a mobile device Or whether the training of that model will happen with unlimited resources in the cloud or whether it's on your laptop without a without a GPU and Depends on the structure needs of your data. So what kind of databases your data stored on is it in a graph database? Or is it in a on a Hadoop cluster? Depends on the feasibility and what I what I mean by that is Are you currently trying to figure out a general? general solution to the to the problem or are you like are you in like some prototyping stage or are you in Production and you want to fine-tune and get the tweak the level of the last little bit out of your model right and Obviously it depends on how much data you have like if you are Training a model with Amazon's clickstream over the last two years Or you train a model with your grandmother's cookie recipe collection the amount of data might Might be a different depending on your grandmother And depends on lots of other things Like how much time you have how much money you have basically how big the team is etc. right to give you like a Some insight on how I approach this whole thing I Let's talk about tensor flow a little bit everyone anyone has not heard about tensor flow by now Everyone it's by far the most popular deep learning framework with more than 100,000 stars on github and what I did is I had those for Features if you will or for categories where I ranked where I gave basically marks for for all these technologies Where ones being the lowest and fires the highest and I think the I mean the most obvious one is probably the ease of setup here with the two and Now the the question is why is that and what I did was I went to the Installation page of tensor flow how you install tensor flow on jubuntu and That's what I got In order to take this screenshot I had to install a browser plug-in that takes a screenshot scrolls down and like stitches all these pictures together and Now you're asking if one is the lowest mark that that Framework can get why is it still to I did the same thing with with the Microsoft cognitive toolkit and This is what I got then in in Microsoft's defense. This is installing it on Ubuntu Apparently installing the Microsoft cognitive toolkit on Windows is rather easy But maybe one quick question is anyone doing machine learning on Windows good to know So to be true to tell you the truth this slide was made about a year ago and The especially in in machine learning everything is changing so fast Tender flow has drastically Reduced the the steps you need to take to install it in fact right now. It's Installed tensor flow Then you have the CPU version So that is that but the side was too funny to skip it Right There A couple of other things So even even last year when when this was the case and you had to go through all this pain to install it Why was popular? Why was tensor flow so popular back then? And that's a couple of things right it has this thing called tensor board, which is basically Real-time visualization of your training process Other frameworks have picked that up as well But tens of we had that from I think the first version 0.6 on Which is kind of cool because you can in real time see how your how your model behaves or your accuracy is your Cost function And it also is able to visualize the computational graph that you bought There There is however There are a couple of things So two other features that I had look at this the degrees of freedom or the parameters exposed basically How many hyperparameters can you play with how many how much in detail can you go to to fine-tune your model and? tensorflow has a ton of options there But that also means So that's great for for this However, it also has like pure tensorflow has a pretty steep learning curve which benefits the ease of use a little bit right and I've already mentioned the the adoption in the community which Which itself? I mean maybe in high adoption rate doesn't particularly say something is good But what it what it can indicate is that if you run into problems and the community using that technology is rather rather large Then you have a high chance of getting actually some help from the community that's That's using it Right same thing I did for Keras where Keras is we've heard about it. Who does not know about Keras? All right, so I quickly talk about Keras is not actually deep learning framework It's something like a higher level interface that you can put on top of Deep learning frameworks and it offers a more intuitive set of Instructions, so you can easily build the neural networks and So for example, you can build a long short-term memory network in like ten lines of code Which is which is pretty crazy and it offers due to that it offers the fast prototyping and You can explore solutions and build PLC's and Keras currently supports a max net deep learning for J tensor flow Microsoft cognitive toolkit and piano and in fact If you install tensor flow, it's already built in and you can use a Keras even without separately installing it The installation is basically if you don't use tensor flow where it comes Default comes by default. You can do the pip install Hands the floor and then you get that however for the other cases you need to have a second separate what they call back-end the The deep learning framework where this Keras then works on top of right That was about the methodology one thing I did is I separated Classical machine learning Libraries or or toolkits from deep learning mainly I mean for several several reasons the resources the resource requirements are different All deep learning framework support GPUs and after all if you can find a linear model to find your To solve your machine learning problem. I would advise picking Is for straightforward model rather than going for deep learning approach Because this is a more complex system and in general If you can solve it the easy way easy way why why not? right This is where I plotted These machine learning libraries against each other where the adoption rate is on the x-axis and the degrees of freedom on the y-axis And the ease of use is displayed by the size and you on the on the on the very right of the chart You can see weaker pie brain and shogun these haven't been particularly popular in the in the last time In fact, pie brain. I don't I think the last change on the github repository was like two and a half years ago I mean, maybe there were five trains in the last two and a half years something like this So I would consider that being being discontinued Weakah offers a graphical user interface which also doesn't seem to be that popular It is however popular when it comes to data mining But I think the the what I would call community kings on the on the left side of the chart There's scikit-learn, which is again, I would assume most of you have heard of highly adopted in the community of us a powerful interface to To play with and you can treat your metal model with the many different parameters and then if you're If you're working with spark there is And there are these other three that that are interested that might you might be interested in ML lip which comes with spark by default and even has a higher level rapper spark ML And H2O which in some cases offers better results than spark ML or ML lip, but then Even if you start with ML liberal spark ML, you can just add H2O later on anyway another Perspective when choosing a machine learning technology, you might have is scalability and we have as I mentioned before scikit-learn is highly doubted and offers a Vast options for improving your model It is however limited to run on a single machine That doesn't particularly mean is it's one thread or one process because some algorithms can be implemented and to run a parallel and as far as I know scikit-learn has done that However, if you aren't if you can't like batch process your your training then you might have a problem because you're limited to the machines memory But if you don't need that's the amount of scalability and you don't need to get no need to go distribute it and you don't Have more memory than it fits more data than it fits in memory and you're comfortable with Python then scikit-learn is is a pretty safe bet There are however H2O ML lip Pachi Mahoud and Chogan, which if you are more comfortable with Java They would they would work quite well and in fact Pachi Mahoud ML Lib and H2O Can all run on top of an HDFS So I guess the the question you have to ask yourself is one of the questions that you want to ask yourself is whether you need distributed computing power and Well, the structure needs of your data being whether it is running on HDFS or not in In fact, we talk about scalability here when we talk about deep learning frameworks that I will come to later They are all highly scalable And by later I mean now And there are a lot of a lot of deep learning frameworks and as I As I said if you know, I didn't say that actually but all of these are capable of Mostly capable of of running the same things that you have seen that you could do with the non-deep learning Libraries that I've shown before It's just the case that in some areas deep learning is significantly more successful and Some examples would be I think computer vision is the prime example of Where deep learning shines a lot and out performs every other approach speech recognition is another example audio recognition and Machine translation as well, but there are there are disadvantages to to deep learning It's more complex The setup might be a little bit more difficult than for these for these frameworks In general you need a little bit more data So there are there's something called transfer learning where you can train your model Take And retrain it slightly Right, so I've only seen this this transfer learning being applied to thank you very Okay, I've only seen transfer learning being applied to Image recognition. So if you have structured data I'm not aware that there's any I don't know pre-trained structure data model that fits your financial analysis Another another drawback of deep learning is It's difficult to comprehend The so what I mean by that is that the the per feature importance It's difficult to figure out Given right it's difficult to figure out. So that means You put in some amount of data and then you get some output Figuring out which of these particularly data points are now responsible for creating that output is It's rather difficult And I I can't think of an example where that would that would make me personally uncomfortable This is great I said I go to a doctor and I Tell the symptoms and the doctor is typing the symptoms into the machine and into a deep learning model and a deep learning model test We have this condition. I Would like the doctor to be able to verify why the deep learning model would suggest I have this condition now and Yeah, the basically the simpler the model is the more easy is to Get a direct relationship between a variable and the outcome Right, so what do deep learning frameworks probably all of you have seen your network by now, so this is your network Artificial your network We have some input nodes and in this case they're all connected to the next layer to the nodes of the next layer And then there's another hidden layer then output layer which Gives you the That you want as Mentioned before deep learning would just have more of these hidden layers I mean This network is rather small. I'm not sure what it could do So if you have an actual model, it would also have more nodes not only more layers But then the question is why do we need deep learning frameworks? and all of these deep learning frameworks offer the basic building blocks of Deep neural networks, so you can just you can get you could just say give me a convolutional layer and then you get a convolutional layer and you can basically create all these layers You so they are I mean all these nodes to some kind of computation And all you can get all these these algorithms You can use different functions that the the deep learning model Provides and all of them are optimized I've said it before the all of these support GPUs, which is I mean The fact that we started putting all these All these computations are similar at the fact that we started putting them on on single shaders on GPUs It's one of the reasons why deep learning has been so successful over the years over the last couple of years and can in fact reduce Training time from days and weeks up to down to hours Right So these are these are the framers you've just seen again mapped against each other with the adoption rate is on the x-axis and the degrees of freedom on the y-axis And the ease of setup as the color according Accordingly to this thing here and the size represents the ease of use and We've I've already talked about tensorflow and Keras They are the most adopted Currently followed by by PyTorch and especially in combination where you have tensorflow back-end and have Keras on top of that They've been highly popular Keras as mentioned is good for this prototyping and you can like quickly define your your model Where tensorflow Absolute outchines every other framework when it comes to deployment whether you deploy it on a distributed system or on mobile devices And if you use both of them you get the both best of both worlds However, there are a couple of things so Debugging and all these fine-tuning and whether you want to run you can define to run on different GPUs all of that would still need to be in tensorflow directly and you can't do that through Keras So in the middle, there's this this Blob of frameworks, which I would consider they're all fighting for attention and and the reason for that is So there are quite a few big companies who have fast amount of data are behind some of those frameworks Data itself isn't Particularly valuable if you don't if you can't get insights out of that data and you can't analyze it correctly and therefore All these companies like Facebook, Microsoft Google Open sourcing these these frameworks Basically make it open for the community to to improve upon is brings value to at least to them All right, there is I think the so I call Keras and tensorflow the the community kings right now, but the the strongest contender Currently is pytorch pytorch is a re-implementation of of torch and pytorch is Being has been it's being developed at at Facebook PyTorch is especially popular amongst researchers pytorch seems to pick up recent academic research results significantly faster than all the other ones and And that seems to be quite important for research. It also has I don't I don't want to go into detail there But it has the dynamic Graph creation which tensorflow now also has I mentioned I mentioned earlier that when I made the first tensorflow slide and that is a little bit outdated because the Tenderflow change a little bit last week tensorflow 2.0 was announced where they have quite a couple of changes and quite a couple of implementations that actually take away the Advantage from from pytorch And then we have a couple of other ones Tiano and I indicated that by this arrow. Tiano was one of the first deep learning frameworks was developed by the University of Montreal, but last year they actually stopped that project because they thought that the the environment the landscape has already matured enough and Piano isn't necessary in that case anymore Then there is Kathy which has also been around for quite a while the It is there is Kathy too, which has also been developed by Facebook Kathy is still around because it seems to be quite popular for image recognition. However you For Kathy you define your model in YAML and that doesn't seem to be up to date anymore I mentioned pytorch towards the original has a very As a fairly small community Maybe because it's written in Lua and you have to write Lua to use it There's deep learning for J as the name suggests this I think the So that was that one over you there is a there is a different again a different point of view You might want to ask yourself when choosing a tool and that is which project life cycle Which part of the project life cycle you are in? There are some of them offer easy fast prototyping You can build your POC is fast Where others offer very fine-gain fine-grained control To basically treat your model and get the most out of that and Machine learning or if you try to solve a problem with machine learning the path to success It's not always that obvious like if you just I mean if you develop Number software and you get your user story You kind of know okay after this this this go to that class to this and then you do that and then your story is done with machine learning You have to experiment a lot. You have to figure out which is the right model I mean there are some there are some ideas if you If you do image recognition, you probably want some convolutional layers in there but in general there's a lot of exploration and and Try an error basically before you know which general model works good for your use case So in the beginning you need a you need the technology that supports this kind of fast We are in later stages of the project life cycle you want something where you can get the last bit the last 0.01% accuracy all of your model and now Mean I'm basically telling you you have to start writing everything like with the framework on this side And then six weeks later throw it away and they start writing the same thing with in the different framework more on the fine-tuning or control side Luckily, that's not that would be horrible. Luckily, that's not the case There is something there are Framework interoperability solutions being developed right now and one of them is on X Stands for opening your network exchange, which is basically a Common format of machine learning models that multiple of these frameworks use So currently there's cafe to Microsoft going to toolkit and makes net and and pytorch who store their model in this specific format so what you can do is and I Mentioned Facebook before because Facebook is Developing pytorch and cafe to and that's exactly what they're doing. They use Pytorch to to design their model Because it's easy you can fast prototype and then take that model The exact same model throw it on to cafe to and then let that run in production and there is Right and not only are these these four that I've mentioned supported but also Like tensor flow for example was not part of them. It's not storing Their model in this specific format. However, they are Converters that are able to convert Tender flow models to on X and I think it's bi-directional. I would guess so right Another perspective I'd want to have when choosing deep learning framework to Solve a problem is mainly a question. You need to ask yourself. Are you solving a core problem? or are you solving a peripheral problem of your domain and Let's say Let's say the you are online retailer and Product recommendation is probably one of the most important things of your business And if you can if you can make your product recommendation 0.1% better. That's I mean that's cash, right? That's probably depending on your I mean depending on the size that can be millions of dollars In that case you do want the the most fine-gain control you can get for for this particular model because it's It's cash, right However, if you and let me give you a different example Imagine you're not an online retailer, but you're a dating site and In order for users to sign up for this dating site you need to they need to upload an ID know for well for multiple reasons people don't like uploading their IDs to some website and We have like 50% of the pictures being uploaded is just not ID at all it's like pictures of I don't know flowers of the wall of something and The verification verification process for these IDs is currently being done manually. So you have like five people verifying these IDs and half of these pictures they look at rubbish Now having a machine model That can filter out 80% of these non ID pictures That would be great right that would reduce your labor and cost a lot but Do you really gain a lot from like if you spend like if you create a 50 people team trying to optimize This use case probably not So a question you have to ask yourself is are you solving a peripheral domain or a core domain problem? and So a couple of things I haven't talked about are these I mean I'm not actually I'm not actually talking about GCP and Azure and AWS but all of these cloud providers have Some machine learning as a service Solutions available, but they are very specific. So you can it usually is like an API where you upload something And then you get a JSON bag or like XML or whatever With the analysis of whatever you've uploaded you can upload pictures and then you can do something like a landmark Detection or you can do face detection or a speaker recognition and all these all these kind of things But you usually can't train The model yourself so you have to you can do some things But you have significantly less freedom as if you develop your machine learning or your deep learning solution yourself but then again if I can just upload All my pictures and Google tells me this is an ID or not my problem is solved right so I mean as long as I'm fine with uploading IDs to one of those services and paying money for that All right Let's skip that actually Right, I hope I could give you like some idea of what the machine learning landscape currently looks like and I hope I could make the point that you need to ask yourself a certain set of questions to make a reasonable decision on How to pick a machine learning tool? Thank you very much. Thank you very much Ico we have time for a couple of questions Hi, Ico. I'm a mode and first of all really thanks for a very nice presentation I really liked your slides and we are presenting is really awesome Regarding one of my question that since being a machine learning developer. I have a question that basically when we are Kind of working on GPUs, especially which library or which kind of API Which we should prefer because basically it should be like Python and kind of tensorflow or Python or open CV Or is any other GPU which we should kind of basically use which is freely available on the cloud? What's your thought on that? Well, I think again it depends on on what situation you are in whether Like I would even argue that not for all cases Especially in terms of prototyping you need a GPU But then as far as I know When when the let's talk about deep learning because then that's where GPUs become interesting As far as I know they all work on CUDA And therefore I I don't see a particularly performance difference in most of them If you I mean if you need The highest possible performance, I think AWS has a couple of of images where you can just like get some ml Instance and then that probably you can scale it up to Whatever your bank account can pay for Further questions if that is not the case, let's thank again Ico very nice presentation