 Hello everyone. My name is Dragan Djuric and this is more closure, less complication. Here's my contact details in case you'd like to find out more. I'm a professor of software engineering at the University of Belgrade. I've been using closure as my primary programming language since 2009. And I teach it since 2010. You can find me on GitHub and you can find lots of related closure articles on my blog Dragan.rocks. Also in the past couple of years I have written a couple of closure books that use libraries that I develop to teach you some novel programming and artificial intelligence topics. Today we will mention a few of the high performance software libraries that I develop. We will walk through a Hello World example that illustrates typical neural networks closure and see how closure makes even more advanced stuff simple, if not always easy. So we all love good powerful software. We love elegant closure software even more. But what I think is even more important is to know how to properly use it in the right context. That's why these two books came to be. Both deep learning for programmers and numerical linear algebra for programmers teach applied math, machine learning and high performance programming in detail, in closure, but they go beyond that. They teach the context and the fundamentals and that is their major mission to open these exotic carriers for typical programmers, preferably using closure of course. And they are primarily supported by a couple of closure libraries that I have written. This is Neanderthal which deals with vectors, matrices and linear algebra operations around these data structures. Also deep diamond which covers tensors which is more or less n-dimensional vectors with all kinds of internal structures optimized for deep learning operations. And this is in line with closure philosophy of using the right data structures and a small set of operations that operate on these data structures. Okay, small number of operations is relative to the domain. It could be large numbers in absolute terms. The point is by combining a limited set of data structures and a limited set of operations we can get a lot of stuff done in a relatively simple way. Also some libraries deals with GPU programming with CUDA with OpenCL and I use these libraries in both Neanderthal and deep diamond. And other cool domains are covered by some other libraries that I developed. But this talk primarily will use deep diamond. These books fund my work on all these things and make uncomplicated viable in the long term. Many thanks to everyone who's supporting me. The wonderful closure community also supports my work through the Closure is Together Foundation grants. Also the books that I write directly fund all this open source work. Oh yeah, these libraries are completely free and completely open source with closure compatible EPL license. So AI and machine learning have been hot topics for the past several years. Many programmers heard about it. However, most programmers never had an opportunity to learn this. On the surface it looks trivial. You just call a couple of functions and the black box magically does the job. Sure, but it is only after someone else found the solution to that specific problem and created the functions you call. To apply a similar solution to a similar problem requires reassembling that black box using lots of math statistics and other esoteric backgrounds. That early point when there's not a straightforward tutorial to follow is when the learning path becomes unbelievably steep. The literature is either heavily theoretical or too superficial. In these books we connect theory and application by starting from scratch. Building the tool itself from scratch. So all these libraries that I'll show you, I also teach you how to create these libraries. And as we go, we apply it on increasingly challenging problems learning the foundations one by one as we need them covering the full path from theory to the actual code that we use. And here is how a simple interactive session of teaching the neural network to predict how sprices based on past values look like in closure. I will lead you through this hello world now to see what are the typical parts of a neural network that does some useful work on some simple data that we all can understand. And I will show you all parts of the code that resulted from using these libraries. Of course in the books I show you all the background and I show you how you can extend this to even features that I didn't support. But you can support them because you will have knowledge to do that. So typically we start from some data source that is out of our control. It might be a web server, it might be a database or in simpler examples it could be a CSV file with serialized data from some other system. So we read this CSV file using plain closure. There might be specialized tools for the task but they heavily enforce the keep it simple and stupid principle in these books by introducing something only when it is needed. So the reader understands not only how but why it is done. So for this data source, plain closure is completely enough. The data records 14 measurements for each of 500 areas in Boston including zoning, age and median house value. Our task is to create a machine that predicts median value based on other data for future cases when we don't know median value and we would like to predict it from other data that we do know at that point in future. So this kind of problem is called regression. Other popular problem class is classification but this example doesn't deal with classification. It deals with regression predicting the actual approximate real values. Neural networks work with numbers. Luckily for this data the transformation from strings is straightforward because the data has been serialized in strings and we loaded it in closure data structures, closure vectors and sequences full of strings. Now we have to convert this to numbers first and luckily in closure this is not a big deal. Note that this part is not accelerated in any way. If we have used a specialized library it might have been faster than playing closure but this is still relatively slow. We work with files and strings. So avoid doing that as much as possible. So only do these conversions at one point and prepare the data, put it in the right format and then only use these formats. Don't go back and forth between playing closure data structures and optimized vectors and matrices and tensors and vice versa unless absolutely necessary. This is usually necessary at the edges of the system because your data comes in unoptimized form but once you convert it don't turn it back at each step. Now it's time to put these numbers into the appropriate high performance tensors meaning data structures that are optimized for internal hardware and that will enable operations to use these stuff optimally. So deep diamond and Neanderthals operations are optimized for these native formats supported underneath these matrices and tensors. On the surface these are all matrices, vectors and tensors but underneath they are much more complex structures so this data is not linearly ordered in actual memory. Underneath there are lots of optimizations that hardware vendors provide us. So the data needs to stay in this context. Don't forget that, don't do unnecessary conversions. Creating the network is technically easy as you can see on this slide. Since I use a closure sourceness to implement a very nice API notice that this is not a specialized data domain-specific language with a magic compiler that will somehow turn these elegant expressions into something that actually executes. Everything here is a closure function and each part can be evaluated and debugged on its own and combine on its own in different parts of software the same as closure. This is not a special syntax. This is just a closure vector with a couple of closure functions in it. So the deep learning for programmers book explains what to do in detail and when to do it and how to do it and the training can be fully automatic as we use here. So the network is unleashed on the data and it learns how to guess that kind of data well. One hint about why neural networks were difficult to apply for a long time even though this is a small network it does have 5,000 parameters that it needs to learn from the data and everything is learned through many, many iterations of trial, error and refinement. So these are millions and billions of operations. If we have tried to write this with our own loops and recures or for loops or every other typical operation that we use for normal software development it would run for days or years even for not so complex examples for this simple example it could be a couple of hours even but with this optimization it would be completed in less than a second. But now when we grade human students we usually don't value how well they compare the text from the book back. We give them new problems which are not too dissimilar from what they have seen before but not completely the same and we see how well they perform. So it's the same with machine learning. We save the portion of the original data which the network has never seen during training and we use that for later testing how the network will perform. So that's one of the typical newcomer's mistakes that they teach a neural network to approximate some function because neural networks are basically functional engines that learn how to approximate a function. So if we know the function there is no point of approximating it but the point here is that we don't know the function and the neural network will discover that function on itself so we have to test it with some other data that it has never seen to see how well this approximation goes. So that is the point of machine learning. It works on new data without explicitly being told the rules as opposed to traditional programming where the programmer decides on the rules and codes the rules in the software using branches and looping and whatever explicit logic that is available in the programming language but here that explicit logic cannot be known because the problem may be too complex so the neural networks learn everything from the data. So the key advantage of neural networks compared to other machine learning techniques is that they do not require much preparation and they do the analysis more or less automatically. Other techniques usually require more human involvement in tidying the data up even for such simple examples but that doesn't mean that the neural network is just some magical black box that can learn everything. It is still up to the programmer to decide on the architecture of the network and to fit it to the problem itself because machine learning is fragile. So the solution only looks simple but it's easy only because it has already been solved. However, it cannot solve your problem. You have to find out a network architecture that works well for your specific problem and that's guesswork. Clojure is great for interactive prototype though so all these can be pretty much the same as we develop normal Clojure software with lots of repl work. On the other hand, even serious networks don't look much more complex in deep diamond. Here's an example of a convolutional network. It automatically fits most arguments of these layers so the resulting API is that simple meaning the user only has to provide very few arguments that the network cannot set by itself. As you can see in this code example, this network is much more advanced than the previous network that we have seen but the code is pretty much on the same level of complexity. To train serious networks, we need accelerators such as graphical processing units or graphical cards. But look at the code. The same code that runs on the CPU in deep diamond will work on the GPU too. I had to use some advanced Clojure internally to achieve this but from the user's point of view it's just regular Clojure. No magic here. And a knowledgeable programmer could point out that what I've shown you is all fine but the latest neural network architecture requires branching and gradient support for that branching and complex graphs. And yes, deep diamond now supports even that although it's still in the snapshot so it's not released yet to the Clojures but you can use it. You can play with that and send bug reports. And here is an example of branching a tensor into several parallel sequential networks for different channels followed by a concatenation into one tensor. From the user's perspective everything works automatically. No need to manually connect all these connections between layers. The point here is that all this is implemented in regular Clojure. I did not develop any magical compiler domain-specific language or magical black boxes. Any competent Clojures can dig in and understand how something is done, fix bugs and create custom layers. They can reuse existing parts of deep diamond at several levels of abstraction for their own specific purposes while each part can be interactively used on its own. Even without the network and surrounding functions such as train, etc. Everything is within reach of the competent user. And any competent Clojure programmer? You're joking with us, you'll think. This surely required a large team of programmers or at least a few programmers and resulted in a million lines of code or a couple of hundred thousand. Well, not at all in Clojure. I created this in my spare time and everything including the integration with verbose, low-level libraries provided by Intel and provided by NVIDIA and direct support for GPU and direct support of custom coding and all automation and forward-looking infrastructure for future extensions is less than 9,000 lines of code. And I'm just a typical Clojure programmer. Some Google genius or anything like that. So if I could do it with a little bit of effort and learning you could do at least part of this if you need it and probably some of you could do this job a lot better than I. So I encourage you to explore this exotic and interesting kind of programming that is more and more required in modern application because now we have lots of data and it's time to analyze it properly. So to sum it up, I have shown you a couple of things here. There are some libraries that are very useful in my opinion. Totally Clojure and in line with Clojure philosophy of a limited set of data structures and a limited set of operations and combining these can give practically unlimited possibilities. I have also shown you the books that are very detailed and that are no fluff, just practical stuff connected to the theory. Nothing is skipped. Everything is in tutorial fashion, not as a reference. So you can learn from this. Also there's lots of free articles at dragon.rocks. By the way, the libraries are completely free and open source and you can use them as any other Clojure libraries. And the books I had to make for pay because I had to fund this work somehow. And also not to forget, Clojure is together foundations who generously support my work. So check them out and if you have interested the work that you need funds for it's always a good time to apply because they're very friendly and very supportive. So thank you for listening and please contact me if you have any questions or advice or anything else that you want to share. Thanks all.