 So here we go with the recording and welcome to class. Okay. Well, welcome everyone. So I just, this is actually Alfredo's lecture today, but I just wanted, since this is our first session, I wanted to just take a few minutes to introduce the course a little bit. Alfredo will give you more details about the plan for the whole course and the mechanics of it and a lot of, you know, issues of tools and software and things like this. And we'll clarify things that I explained during the course that you may not understand directly during the lecture, but Alfredo will make everything clear. I thought I would talk a little bit about the history of this, right? Because this is not something that we talk very much about during lectures. And deep learning really is the descendant of work on neural net, which itself is the descendant of work where people try to understand how biological systems kind of organize themselves, but particularly self-identals. So going back to the 1940s, two gentlemen on the right, McCulloch and Pitts came up with the idea that if you connect very simple computing elements that are essentially what we now call McCulloch and Pitts neurons, binary neurons, you can do logical operations. And if you can do logical operations, you can do reasoning. And so they had this hypothesis that essentially the brain was a very large, this is not a hypothesis, it's a fact. The brain is a very large network of neurons. And those neurons are actually somewhat binary in the brain. They either turn on with a spike or they turn off. And so they say if you have binary elements, you can compute, you can do logical calculus. You can do inference, you know, logical computation. And perhaps this is a process that the brain, the human brain uses for reasoning and perhaps to bring in animals. So very soon, so simultaneously Donald Hebb, who is a psychologist, came up with a proposal for how neurons in the brain by changing the strength of the connections between neurons can sort of change their function if you want. And so it was known in neuroscience, but the idea that learning proceeds by changing the strength of the connection between neurons is this idea. And Hebb proposed that if two neurons are active at the same time, then the connection, the synapse that connects them strengthens. And if they're not active at the same time, then it kind of gets depressed. So this is called Hebbian Learning. And this was sort of a general idea that a few people kind of reused in the first sort of artificial neural network models. There's a lot of work that I'm not mentioning here, but in the 1940s as well, Norbert Wiener founded a field a discipline called cybernetics. We don't use that word very much anymore. But we call this now system theory or things like this. There's a lot of subfields of cybernetics now, but we don't very much use that word anymore. But it's the idea that we have sort of complex systems in interaction with feedback. You can have sort of very complex phenomena emerging. And so he came up with sort of a bit of general theory for this. And out of that came thoughts about self-organization or auto-organization if you want to be Latin. And this is the idea that if you connect lots and lots of very simple elements with a very simple rule to kind of make them compute, they perhaps might be able to self-organize themselves into having a sort of emergent property. And this is something that we observe with biology and with the brain in particular. The elementary components of the brain are very simple in conceptually. But it's their connection, it's their network that gives emergence to things like intelligence. So that was a very mysterious process back then. Then in the late 50s, Frank Rosenblatt, who also was a psychologist actually, came up with a system called a perceptron, an idea. So it's a very simple idea, which is the basis of what we now call supervised running of basically having a system that can learn to, for example, classify patterns represented as a vector of numbers or activities or voltages in this case by adjusting weights, the weights being similar to the synapses in a neuron. And he was able to demonstrate that such systems that basically you're learned by error correction can learn basic things. This is what we now call the linear classifier or which is an example of what's called statistical pattern recognition. So it's a very simple idea. You show an image to a system. You see if it gives you the right answer. If it gives you the right answer, you do nothing. If it gives you a wrong answer, you adjust the parameters which are weights of a neuron so that the answer gets closer to the one you want. And then you keep doing this with thousands of examples. They didn't have access to computers at the time. So what he did was that he built an analog machine, an electronic machine that basically implemented all this. And that was the physical perception. You see pictures of this at the bottom left. And in fact, one of those pictures is Rosenblatt holding a row of eight weights. The weights were actually potentiometers with a motor so they could rotate. So it's basically an electro-mechanical computer if you want. We made a replica somehow this past semester. We made a replica. Yes, actually, Phillip Schmidt, who's working in a, you know, I've made kind of a replica of this, but with modern technology it's much easier. Then almost simultaneously, Bernie Riedro as Stanford came up with a slightly different model called Adaline, which is very, very similar in many ways. Also basically a classifier. And around the same time, neuroscientists discovered some basic properties about the architecture of the visual cortex, which turned out to have a big impact several years later, which I'll come back to. So they actually got a Nobel Prize for this. So this was very well known. I mean, this became very well known and they discovered that neurons in the visual cortex basically look at kind of a small area of the visual field. And many neurons at different places in the visual cortex basically perform the same operation. The third picture from the left is, at the bottom is Huber and Riesel. Then some work on all of this in the 1960s. And then in 1969, Minsky and Papert from MIT published a book called Perceptron that actually pointed out the limits of the Perceptron. And they said, you know, trying to build intelligent machines from the basic idea of the Perceptron is basically hopeless because of those mathematical limits. And so really, that's not the right way to go. We should pursue the idea of intelligent machines in some other way, okay? And that basically killed the entire field. So people stopped working on these things in the late 70s. They didn't really stop working on it. They changed the name of what they were doing. But people in AI who are interested in sort of reproducing the process of intelligence, if you want, started working on things that are more based on logic and sort of more classical computer science. What was not then yet classical computer science but what became kind of classical computer science. Algorithms, search, logic, you know, things like that. Reasoning and sort of abandoning all the kind of, the type of things we were observing animals. So this is, again, more pictures of the Perceptron at the bottom. Now you can reproduce the three pictures at the bottom here with about, you know, half a page of PyTorch. But back then it was kind of a major undertaking. And then on the right here is Bernie Ridgill with the original Adeline, which was also an analog computer where the weights were electrochemical, actually. And the basic element in those were the macular pitch neuron that performs a weighted sum of its inputs where the weights are adaptive and then compared it to a threshold and turns on if it's above the threshold, turns off if it's not. So a little bit more history of what happened afterwards. So basically research on neural nets and this kind of stuff kind of pretty much died in the West at least in the 1970s. There were, so people kind of changed the name of what they were doing, you know, that became known as statistical pattern recognition. And I'm missing an N here on pattern. Or for some others, Bernie Ridgill, for example, renamed what he was doing as adaptive filters. And this turned out to have actually a huge impact on technology. Adaptive filters are used absolutely everywhere today. And for data communication for, you know, every modem has an adaptive filter in it for equi-cancelation. So for example, right now, if Alfredo talks, the sound comes out of my speakers and it goes back into my microphone. And if it wasn't for equi-cancelation, you would hear his voice twice. But equi-cancelation is basically an adaptive filter that learns to suppress things like that. So you can't have speakerphones without that. You can have video, interactive video without that. But in Japan, research, which was kind of a slightly separate ecosystem for research, research continued in neural net. Kunoiko Fukushima came up with this thing called a cognitron and a neocognitron, which inspired a lot of people, including me. I kind of started getting interested in neural net just around the same time, around 1980. And I was reading from us by those papers. He got inspired by Hubel and Riesel and tried to kind of have a sort of computer simulation, if you want, of the visual cortex, according to the model of Hubel and Riesel. He didn't have proper learning algorithms. So his system was a little brittle. But that was kind of a pretty heroic attempt. And then in the early 1980s, the physicists started getting interested in neural nets. And the physicists really didn't know the whole history of the perceptron. And so there was no kind of bad feeling about kind of working on neural nets, like engineers and computer scientists had who knew that the perceptron basically had been killed. And it was a bad idea to try to publish papers on this. Physicists didn't have those hesitations. And so Hubel came up with a model which actually had been proposed earlier by other people. But he came up with a theory for it. It's a type of recurrent neural net that he showed could be used as associative memories. Again, other people had shown this before, to over-cohono in Finland and Nakano in Japan. But what he showed was that there was a mathematical analogy with systems that physicists were interested in called spin glasses. And that got physicists interested in basically trying to use their methods from statistical physics to explain perhaps how the brain works. And one of them was Terry Sinovsky. He was actually a student with John Hubfield. For what he tells me, he's the one who actually got John Hubfield interested in neural nets. And he worked with Jeffrey Hinton on a model called the bus machine. When I was working on this, I found this paper fascinating because this was really the first paper in my mind that allowed a system, a neural net, to be trained that had hidden units. So the main limitations of both Hubfield nets and Perceptron and Adeline on all the previous models was like there were basically single-layer systems. Every unit is either an input or an output and you only have weights between inputs and outputs. And really the limitation that Minsky and Pepper showed one of the limitations is that to really have a powerful machine, you need multiple layers. Also machines were the first system, the first learning algorithm that basically allowed training with units that are neither inputs nor outputs. Do you have a question? Okay, no question. Then pretty soon the idea of back propagation came up and it became practical to basically train multilayer neural nets with back propagation. This required a major conceptual change which was to switch from using binary neurons to basically continuous neurons so that they were differentiable, essentially. And a lot of what we talk about during this class is really related to this idea of back propagation. So that really was a watershed. It spurred a lot of interest in the field and a lot of the community became fairly large. The NIPPS conference now called NORIPS kind of started around 1987. So this really kind of started the field of neural net, restarted the field of neural nets. A few years later, I proposed the idea of convolutional nets which is basically inspired by the Neocognitron and by Hugo and Riesel. But it uses back propagation for training and that sort of allowed things like, you know, handwriting recognition and things like this and more recently image recognition. Then there's a number of progress that occurred which may or may not be on everybody's radar but they are important to me. Leon Boutou, who's now at Facebook and Patrick Gallinari in Paris, proposed this idea that you could basically do automatic differentiation. You could build a neural net by assembling modules and if you knew how to propagate gradient through those modules, you could basically compute gradients for any kind of graph of interconnected modules. And Leon Boutou soon rejoined Bell Labs where I was working and we started implementing a software which was called SN at the time that actually implemented this idea around 1992 and we used this for all of work and it took a long time for the rest of the world to actually realize this was a very useful thing. So what you see in PyTorch today and TensorFlow and various other schemes are essentially derived from this idea, if you want. And SN means? SN means simulator neural net. So this was sort of a French name that we gave to our neural net simulator. And the paper in French. Really open source in 2002 under the name Lush. It's on Sourceforge if you are interested. It has a list language at the front end. In the mid-90s, there was interesting work on recurrent net like O'Reiter and Schmiduber's long short-term memory and various other things. Interesting work also in speech and handwriting recognition on what's called structure prediction with neural nets, graph transformer nets, which we'll talk about a little bit in this class as well. And then nothing happened for about 10 years. People lost interest in neural nets. People in the machine learning community started being more interested in sort of simpler models that basically were single layer and for reasons that are still mysterious to me. And nothing much happened. A few things. Joshua Benjo came up with this idea of neural language model in 2003, which turned out to be really pioneering. There's a lot of work on this. If you have heard of GPT-3, this is basically the descent of this if you want. And a lot of models that use NLP today, the neural models for NLP are basically very inspired by either this or the last item here, the Colorban Western Self-Supervised Neural Net for Natural Language Processing. This is also a very pioneering paper. 10 years ahead or 15 years ahead of this time of basically training a system to represent text by training it to discriminate good sequences of words from bad sequences of words where you have substituted some words by others, for example. They won the 10-year award on new reaps, I think. They did, yes. It was that last year, two years ago. Two years ago, I think. Because their original paper was actually earlier than 2010, their journal paper, I think, was around 2010. But then in the mid-2000, Joshua Benjo, Jeff Hinton and I kind of started a conspiracy to renew the interest of the community in neural nets. And out of this came out this idea that you can sort of pre-train a deep neural net, perhaps in a layer-wise fashion, using autoencoder methods and things like that in an unsupervised or self-supervised manner. And that sort of restarted the interest of the community a little bit into neural nets, but only kind of a small community until around 2012 when it became clear that convolutional nets, when you run them on GPU and had large number of training samples, basically could beat every other technique that were around. So that was really a watershed moment. A lot of people became interested in deep learning around 2013 after this AlexNet won the competition. So this was from Jeff Hinton's group, Alex Krzyski, Elias Hitzkever, and Jeff Hinton. We also had been working at NYU on convolutional nets for various things, but we were not as fast to implement things on GPUs. So our applications were good, but not as spectacular. And then three years later, Elias Hitzkever, who was working at Google at the time, he now kind of leads OpenAI Research Branch, showed that you can use neural nets, multilateral STM in this case, for neural machine translation and basically match the performance of traditional machine translation system. Simultaneously, people at Facebook Air Research, which was just the beginning of Facebook Air Research at the time, that just existed for a year, Western Entrepreneur Board showed an idea called memory networks, which basically said like you can have modules in a neural net that are basically associative memories. A year later, Barano Kunyum Cho, who is a professor here at NYU in Yoshe Benio, showed that you could replace this complex at STM that Elias Hitzkever had shown by a relatively simpler architecture based on the mechanism of attention, where you have basically multiplicative interactions between weights that essentially gate can route kind of signals if you want in a complex neural network. And when you combine the memory network idea with this idea of attention in sort of a big way, you build big neural nets with that, you get what's called a transformer architecture, which has basically completely taken over the field of natural language processing and is now actually making in rows as well in computer vision. 2016 was ResNet, which was kind of a modification of convolutional net that really made a big difference, because it had to be much deeper than they were before and really sort of brought about a big progress. So there's a lot more here that is in the blank, more recent history, which you'll hear during the class. And the class will talk about the basic techniques, their sort of practical use and how you can put them to practice. And I wouldn't say the theory in the sense that we're not going to demonstrate proof theorems or anything like that, but certainly a lot of mathematical basis for all of this. The style is that in my lectures, I'm generally going to stay very sort of conceptual. I'm going to talk about things in sort of give you an intuition for how things work and give you the grand ideas. Then the actual math behind it and the practice of actually making it work. A lot of this would be covered in the practicums and also sort of in your personal work. Okay, I'm going to start here. Hold on, there are questions for you. Go ahead with the questions. All right, so you covered like the historical part. What about the other edge? Like what about the future? So what are the research you're working on? What do you think is going to be a good direction? We'll talk about that also in the class. So towards the, you know, in the second half. So one big hot topic at the moment is something called self-supervised learning. So this is really kind of the descendant of this layer-wise unsupervised pre-training for deep network from 2006. It's kind of the same idea that and also inspired by the Calabria Western system and inspired by some work I did also in the 90s on what's called Sammy's Network. So it's the idea that you pre-train a neural net not to perform a task in a supervised manner but you pre-train it to just understand the data without training it for a particular task. And then you fine-tune it on the task that you're interested in. This has revolutionized natural language processing over the last two or three years. It's starting to revolutionize computer vision and certainly a lot of areas where the amount of data that we have labeled data is limited. So self-supervised learning is really kind of the immediate future. This is what a lot of people are working on at the moment and it's making really quick progress. A lot of work is also being devoted to basically getting machines to reason. So a lot of what I've talked about here is basically perception. What we can do in neural nets is perhaps what animals and humans can do in a very short time without thinking about it. So recognizing objects, navigating things that don't require reasoning. But what about if you want machines to be able to do long chains of reasoning or plan in advance or things like this, how do we need to modify our architecture so that the machines are able to do this? There's a little bit of that going on in other fields around transformers and things like that but not a huge amount. Networks are really an attempt at allowing machines to reason, access a working memory, reason on it and then change the content of the working memory. So a lot of recent work on this. I don't think anybody has a complete answer for how you do this but it's a big challenge to basically try to make reasoning compatible with deep learning which means gradient-based learning really. Classical methods for reasoning in AI are based on logic and that's discreet and essentially incompatible with learning. So that's kind of a big challenge. There are other challenges which are a little outside of the reach of what we can do at the moment. So things like we know how to train a system to represent percepts like images and text and things like this. We don't know how to train a system to represent action plans in a hierarchical manner. And I think that's a major obstacle also. People are not really working on this at the moment because no one has any good idea of how to do it but that's kind of a third challenge I would say. So learning without labels, learning a little bit like humans and animals by observing the world, that's getting machines to be able to do this. That's a major challenge. That's why I'm working on. That's basically the only thing I'm interested in really research-wise. Getting machines to reason is a very interesting topic as well. A lot of people are working on this and then the third topic is basically integrating all of this into sort of autonomous intelligence systems that has an objective, has a model of the world that allows you to predict the consequences of its actions, has a way of deriving a sequence of actions that will optimize its objective and allows you to learn all of this simultaneously. That would be kind of the ultimate intelligence system. We don't know how to put those things together yet. Those are the challenges. Have fun with... Hold on. Last question. These were interesting directions. What are the limitations? What do you think about quantum computers, non-binary states? That's the last one. Current limitations are clear. When I talk about those topics of research, cell supervised learning, reasoning, et cetera, basically what I say now is that current systems are not able to do this with a few exceptions. One big challenge is, can we come up with learning paradigms that would allow machines to learn by observation and would allow them to acquire some level of common sense? No machine has common sense at the moment. All of AI systems usually has a very narrow intelligence. I'm kind of skeptical about the applications of quantum computing to machine learning. It's an interesting field. There are people who are thinking about the connection between machine learning, deep learning, and quantum computing and whether one could be used for the other. It's still very much up in the air. It's not entirely clear. There's certainly not going to be any practical consequences in the short term, at least in the next decade. But as an academic research subject, it's interesting. Okay. All right. Have fun. Thank you again for opening this semester and this year class. We see you on Wednesday, right? At the same time, same place. There is a different Zoom link, but we all have the access to the calendar, so we know where to find you. Okay. Thank you. Bye-bye. All right. So back to us. Yes, welcome again. Let's cover a little bit of following up topics. One of the most interesting topics for you, I think, is going to be figuring out how this course works. I share with you already on the 3D announcement. I share the sort of rules of the class. You should have already have access to that because, again, I guess you're here. So all of the people that are here right now have access, right? But more or less, you see that there are... So maybe we want to cover quickly the grading part. That is what usually you're very interested. So I will respect that. But, okay, before talking about this stuff, right? So as you can see, this class is remote, right? And then, so it's going to be slightly different from how usually we held classes in person. Every interaction is going to be done through, you know, a camera, a computer, and whatever. And so we still need to be able to communicate at the best of our capabilities, right? So as you have already noticed, what I'm trying to do is going to be, like, try to stop Jan as many times as I can given that we have questions for him. Otherwise, he just keeps going and we don't understand... Well, we might lose focus and, you know, understanding of what's going on. So whenever there are, like, good questions, right? No, not just... Whenever we actually have a proper question to ask Jan, we should be writing down on the chat such that I understand what is the understanding and the pace of the class. It's too fast, too slow, whatever. And we can interact, right? So interaction is the base to actually get something out of these classes, right? All these classes are recorded, right? I just pressed the record button. So, you know, eventually after at the end of the class, you'll be able to watch, again, this... This is the competition I'm going to tell you in a sec. So, see, all these lessons are recorded. Therefore, it means you can re-watch them later, right? This does not mean you don't come to class, right? The part that you come to class, it's, I think, one of the most important parts because you can get much more value out of, you know, the content during class and have this kind of interaction. I will try to have eye contact with you over there on the other side of the screen as much as I can. And again, I will be trying to be interactive for what I can, right? And these look seem to be very effective whenever I be using, like, the chat here. And by, you know, I ask to you, you're right here, we somehow have a feedback loop to stay on the same page. But you don't have to turn on your camera because, again, otherwise, if everyone turns on the camera, you don't see any more what's going on, I think. All right. So, Jan's speed of talking was a bit faster. I know. You can watch the video recording and you can slow him down. It was just because he didn't want to steal too much time from the lab, but, okay, this lab is like half gone through. So, let's see what we can do. Will be the work with PyTorch. Yeah, of course. I'm going to be talking about that right now. So, yeah. So, the point is that we should be able to learn, right? The point of coming to class is learning. And to be able to learn, you should be able to keep awake for the most of the class, right? If you, again, if you go around and eat some ice cream because you're home and whatever, right, you may get distracted, then you don't get any more back on track on whatever is happening. Yes, there are recordings, but, again, recordings are just, oh, you're sick one day. Yes, you don't miss out, right? So, recording are like something you may use if you really, really cannot take a part of class, right? But otherwise, we should be able to, you know, make like a happy group, happy interactive group. Also, I'm very sarcastic, okay? I'm just telling this in advance for whoever does not understand or know what sarcasm is. So, what is written down in textual format is usually the official part. Whatever I say, you know, take it with a pinch of salt, as in, you know, I like to joke quite a bit. And so, I will say funny things to make you smile and laugh. So, don't get offended, please. Or like if you think I'm a jerk, just let me know. You think that it's all good. I will try to be more cautious, right? Whenever I interact. Okay, so these are the main points. So, yeah, the agreement is that I will, okay, every time you don't understand something I say, right? Just ask me, what did you say? I didn't get it, right? So, if you don't get something, at least in my classes, it's usually my fault. I'm not able to communicate with you, right? So, my accent is bad. I haven't slept enough. That's usually very often the case whenever I'm overworked. Or, you know, something happens. There is, you know, someone ringing the door, you know, I'm at home, right? So, someone will eventually bomb our lecture, like today, young, perhaps. But, okay, that was somehow planned. Somehow it was not, since I don't even have his life. But, okay, it's fun, right? So, life, it's interesting because it's different every day. Okay, so, if you don't understand, ask me. I will repeat until you get it, right? I'm more, I'm positive that eventually you will get everything I say, right? At least that's my hope, right? I will try my best. All right, first point, done. Second point, how the grading and stuff works, right? That's the part that some of you are super interested. I have to be honest with you in advance. And I say, I don't care. I don't care about the grade I'm going to be giving you. The grade is for you, right? I don't make GPA or things I don't care. So I will not be mean as in I will not, I will, you know, you get the grade that you want. I don't give you the grade I want. I don't care what grade I'm giving you. You should be the one responsible of getting the grade you want, right? And regardless of whatever the class goes, eventually you will get whatever you deserve, right? All right, so this is what your peers have done last semester, last year actually, they were amazing. We are awesome too, right? So you will contribute as well if you want to make this website and course even a better thing. So this is the schedule from last year and everything is recorded. As you can see, there is a video button here. And I spent, you know, too many hours putting this stuff from YouTube. Anyway, so here you have all the video recordings from last year. And then here, history and motivation, you click here and you have the summary of what Jan talked about today more or less, right? From last year. So one easy way to get free points for your final score is going to be finding typos. If you find typos in these write-ups, you get free points, okay? I mean, yes, I'm slightly joking, slightly not. If you really help making the course a better course, then you are rewarded. And everything is detailed in this document, right? So, right, that's the last semester, last year course, right? And you have also translations in, you know, Arabic, Spanish, French, Italian, Japanese and whatever. Because we want to be accessible, right, to everyone. Yes, you get an A with us. You can do research with us, okay? So if you want to join Jan's lab, get an A in this course. Then we can talk about joining the lab, right? But A is a prerequisite to be able to join. Do we use Python? Yes, we use Python and PyTorch for the coding part. Could we get access to this slide prior to class? Yes, you will be able to. I know there are slides before class. Today Jan bombed the class, so I didn't even know he had slides. Yes, usually you will have slides before. And all slides are online from last year. They're all online on the website, right? So, but I guess, yes, I will put the updated version as well. If I know there are slides for a given day. Okay, so again, get an A. We can talk about joining Jan's lab. You have the class website. Okay, so I guess we're going to... We have what time? We have like eight minutes before 10.30. Okay. Any material for books? And okay, I'm writing the book. The book is going to be out in December, so not yet. Does not. You don't need high skills in Python. You just need all you need to do the coding. The coding is going to be explained by me in this course. I never took a Kaggle competition, so you don't... You're not required to know what a Kaggle competition is. So we explain that as well. That's in a second half of the course. Okay. Last part. Okay, today we don't teach. I guess it's fine. Administrative day, fine, cool. So again, since we are all remote, all digital, we are not interacting in person because there is this pandemic going on. So how do I teach? Do I teach only in class? No. I also teach outside class. I'm very into teaching. And so if you're interested to get additional education free of charge and everyone can access this stuff, you can also find much content on Twitter. So if you go on twitter.com and you go slash, you put alpha and then you go C and Z, which is my surname sort of shortening. If you go here, you're going to see a lot of content if the website actually loads. Okay. So you're going to see a lot of content. This is advertisement. Leave it to me. But otherwise, I usually post other, okay, I post some food, but okay, never mind. Usually I post about machine learning and, okay, there you go. So this is like math communication, right? So I try to push, you know, my educational agenda online live, right? I'm into art and education, right? So in here, I was just talking about getting life organized. Okay. This is just the beginning of the year. Sorry. It's going to be a bit noisy. But usually like you have like math content. For example, here I told you, I'll tell you about the latest new version book of Kevin Murphy, which was just released. This is super nice book. But what I wanted to show you here was the first part, which you should really check out. It's going to be going on the search bar. You want to type something like from, and then you do colon, alf, c and z. And then you can write, for example, linear algebra, LA, right? So if you write LA, no, maybe let's do linear algebra, right? Linear algebra. Okay. So here are very nice videos of linear algebra. This is fundamental, okay? If you don't know linear algebra or if you're rusting linear algebra, do check these videos out. This course is based on linear algebra. If you don't know linear algebra, then good luck in this field, right? No, not just with the course. I'm just talking about generally, about the field. So this is really good. It's done by Gilbert Strang. Sorry, my bad. Yeah. William Gilbert Strang, he's just an amazing educator. He's all as, you know, AF. But he's just amazing, right? So just check this stuff out if you don't know, if you haven't heard. Another stuff you want to see, okay, I'm talking about integral operators, you don't care. Another interesting part is this one, right? So this dude here, Grand Sanderson, is another person which is very into education. And he also makes free, accessible content online, on this case, in YouTube. And if you've really done, if you're really, really, really rusty in linear algebra, please binge watch this essence of linear algebra. Okay. Everything I explain in my classes is based on this perspective, right? And so if you want to get started ahead, like how do you say jumpstart the, your, your understanding of all the lectures, you may want to really binge eat these videos here. You also, you can press the heart if you want. That's appreciated. And this is going to really help you for your class, score, career, future, everything, skin. Okay. All right. Jokes aside, check this stuff out. It's, it's, it's really good. Okay. I wouldn't be telling you this stuff. I don't get money out of these things. I'm just trying to be as helpful as I can. And we have three minutes left before the end of the class. Okay. All right. So enough advertisement. I just show you, I just show you a cool video. Okay. And then next time we talk about pytorch and things. Okay. So cool. This, this is appreciated, right? So your, your comments, I like your comments on the chat. So that makes me feel you are with me and I'm not too crazy. Okay. Okay. Again, if you think I am weird, that's okay. Now, if you think I am a jerk, there you go. That's not good usually. So let me know privately and I will try to be changing my something, right? But otherwise this is me. I'm weird, funny, whatever. Okay. I'm very funny. Again, don't try private messages. I can't really reply to all private messages, right? Okay. So last one, last pit, and that was just going to be the end of the class. I'm going to be showing just a pretty video. In theory today I should have, I should have shown what a neural network does, right? And in, in a, usually when people talk about neural nets, the easiest thing to speak, to think about is classification and whenever you want to do classification, when we have classification, basically it means try to sort, well, to sort, that to split things, okay, to sort things in different buckets, right? You have like pictures of a dog, pictures of a cat, pictures of a rabbit. You want to put each picture in the right bucket, right? And so what I show you usually are things like, okay, let's do this. Since we have internet here, classification, right? So you want to be able to use Google a lot, class images, you know, okay, my internet connection sucks. No, no, classification, okay, machine learning. What is this stuff? Okay, there you go. So usually, okay, this is what they do, right? And that's what they show, right? It's a straight line, okay? If it's a straight line, these two classes are called linearly separable, right? This is not usually the case. Usually what's the case is stuff is not linearly separable. Okay, let's see a picture and there is, okay. All right, there you go. Okay, this is all better, right? So these points are not linearly separable, right? So you cannot write a straight line. And so people usually show that you can have these boundaries, which are called decision boundaries, which are like turning, right? So I don't like this perspective because it thinks, it makes you believe that you can turn decision boundaries in a weird way. What I like to do instead, instead of having this representation, I want to warp the space such that that line is still a straight line, right? Because all we have in deep learning and whatever is gonna be a linear decision boundary, right? This is my simple singular module. So when we have a neural net, which we don't know what it is right now, but I just show you now a very cute animation, you basically see how things that are not linearly separable, let's see, non-linearly separable data, there you go. So whenever you have non-linearly separable, like this one, you cannot separate them with a line. Or, okay, a better example. Okay, this one here, for example, you cannot put a line, right? They are warped. Let's say like a spiral, whatever, spiral. Here, so let's say you have a spiral, or this class is here, you cannot write a line through, right? You cannot cut them in half. So how do we get to classify a spiral? This looks exactly like my stuff, okay. Oh, there we go, here. So how do we get to write a line through this stuff? We're gonna be using a network to move the space, and, oh, there we go. So these are the intersections, see? Okay, this is me. So here you have a straight line, but then you cannot cut this stuff, right? So I'm gonna show you this video about how a neural network basically morphs the space such that it becomes linearly separable. And then we're gonna say goodbye, so I don't take too much of your time, extra time. So let me open the video. This was a very interesting not-planned class. And we go with this one. And we should be able to see something. So these were my original five branches, right? So these are not linearly separable, because if you draw lines, you're gonna be intersecting these classes, right? Instead, what the network does is gonna be unwarp the spiral such that I can still draw lines, right? So I'm gonna be showing here this animation. And next time, I'm gonna be explaining how everything is done, right? I'm just showing this for making you hungry, okay? Have you had breakfast? Yeah? I'm just kidding, like, educationally hungry, right? What does morph in the space mean? Yes, next time I show, I tell you everything. This is just to show you some very cool animation. This is the stuff I like, right? Also my high schooler, here's a video so you can find out if you like, if you write from AlfKanz high school, you're gonna find my high schooler video on YouTube as well. So I'm really into this kind of visualization, education, how do you call it? No interpretative. What's called? The thing you try to understand, right? You try to understand how it works inside. There we go. So this is basically what the network did. The network warped the space such that all the points now are spread apart. And then when these points are spread apart, they are spread apart in a specific way. This specific way is that they are now linearly separable, right? You can see, you can draw a line like this one between these yellow points and the green points. This bar is in the way. You can draw a line between the green points and the purple points and so on, right? And what are these arrows? Okay, this is a question for you, right? For next time, let me know what these arrows are. There are five arrows in two dimensions. There are five classes. And we are on a 2D screen, okay? So let me know what you think next time, right? And this is a question for you to think about. And that was basically it. We are over time as usual. I'm sorry about that. I'm stopping the share screen. So all questions that are here that I didn't see, I will not be able to answer. Just write on CampusWire if there are, you know, fundamental questions about the content. I will, we will reply. There is office hours tomorrow from a judging, super smart guy. Feedback on the class, anything. There is the CampusWire chat for the feedback. That's it. Okay, anything else? That's it. I think we are done, right? We are late. I'm late, but it's okay. I don't know if it's okay. I think it's okay. All right. Sweet. Next time, PyTorch, we are going to be looking into this stuff. I will write down the architecture of this network, give you more context, and we'll get a little bit more meat, right? Like Jan took half of the class today, and then I couldn't finish. All right. Bye. And I cannot even close the thing.