 Hello everyone, this is Alice Gao. In this video, I'm going to start talking about artificial neural networks. This is the second algorithm I'm going to discuss in the machine learning unit of our course. And this is the first lecture on artificial neural networks. There is a huge hype on deep learning of artificial neural networks right now. So I thought I'll spend a little bit of time talking about the history and also talking about some backgrounds just to show you how did we come here? How did we come to this moment where there is this topic of neural network is so popular? So going back to a few terms, what is artificial intelligence? What is machine learning? And what is deep learning? And what are the relationships between these? Artificial intelligence is about building machines that can behave intelligently. We've talked about a few definitions of AI already. The idea of AI has been around for centuries for a long, long time, but it was first popularized by Alan Turing's seminal paper in 1950. So that's AI. Under the big umbrella of AI, we have machine learning. Machine learning, the goal is to let computers learn instead of being explicitly programmed. And machine learning primarily uses statistical methods to achieve their learning. Now, what is deep learning? You can see that machine learning is only one branch inside the huge umbrella of AI. And then deep learning is again, one branch of machine learning. So deep learning is about developing these hierarchical networks called artificial neural networks by mimicking the structure of the human brain. So human brain is incredibly complex. And even to today, we don't know enough about how it operates. But since we created these really complicated networks, these models, we can use these models to learn complex and abstract data. So the concept of deep learning, you might be surprised to learn this, have been around for decades. So it's not a new thing that just came out in recent years. However, the idea of deep learning really came into the spotlight and received a lot of publicity in the late 2000. And one reason for the recent success of deep learning is that now we have much more powerful computers and we also have a lot more data. So both of these contribute to the success of deep learning in recent years. There are two events that you should probably know about deep learning. So one is that in 2012, a deep learning algorithm called Alex net won the image net challenge. This caused deep learning to receive a lot of publicity around it and became really popular. This is an example of using deep learning, using neural networks for a supervised learning pro problem. But in the same year, Google brain also made a breakthrough by using deep learning successfully for unsupervised learning problem. So you might have heard of the cat experiment. This is an experiment where Google used a deep learning algorithm to process over 10 million images, random images from YouTube videos. And it turns out in the end for the network, we learned one node in the network developed a very strong affinity for cats. So that node is able to recognize when the image has a cat in it. This is why it's called the infamous cat experiment. I hope the little bit of background history gave you some perspective on why deep learning is so popular these days. So let's talk about why do we want to use a deep learning algorithm, want to use a neural network, consider some applications where we want, we have an image and we want to recognize what's in the image, image interpretation, or we have some speech where, for example, we're developing something like Siri where we want to talk with it and we wanted to help us perform tasks or think about machine translation. These are all applications where the data is extremely complex. In particular, the data is complex because the relationship between the inputs and outputs are very complex. So how do we deal with such complex data? If we want to learn something useful and use our learn model to perform some tasks, well, it turns out we already know that humans can learn these complex relationships very well, right? Because humans can perform all of these tasks, although we don't quite understand how we do it, but we know that we are able to complete these tasks just fine. So if humans are able to do this, well, then can we build a model that mimics the human brain? This is the original motivation for why people consider building artificial neural networks. So what is the structure of the human brain? Well, there's a field called neuroscience which focuses on understanding the neurons and how what's the structure of the human brain. So because of neuroscience, now we have some understanding of what the brain looks like. The brain is a set of densely connected neurons. So it has a lot of neurons. These neurons have a lot of connections with each other. And each neuron is a very, very simple component, a very simple unit, but it still has a few components. Let's label these components on our graph down here. So we have a picture of neuron right here. And then the neuron is going to receive inputs from other neurons. So these parts that you can see, these are where the body of this neuron receives inputs from other neurons. These are called dendrites. Then we have the main body, the cell body of the neuron, which is called the soma. So the soma controls activity of the neuron. So if we think computationally, the neuron is going to receive input signals from other neurons. These are inputs. And then the soma is going to do some computation, right? Some really simple computation. And once the computation is done, the soma is going to decide whether to send an output signal or not to other neurons and how strong is the output signal. So it's going to send the output signal to other neurons via something called axon. Finally, we have a general term to describe the links between neurons. This is called the synapse. Now don't worry too much about these terms. This is not a biology class and certainly not going to test you on how well you memorize all of these terms and whether you can label the picture. And as you can see, I'm not labeling the picture very accurately, but I thought it's important for you to understand the basic structure of a neuron so you understand why we would come up with a corresponding mathematical computational model for the neuron. So as I mentioned, the neuron receives input signals from other neurons. And then depending on these signals, it's going to perform some computation, usually very simple computation. And then it's going to decide whether to send an output signal or not. And if it sends one, how strong is the output signal? Something interesting to think about here is that if we convert the structure of a human brain to a computational model, this computational model is actually quite different from other computational models that we've talked about so far. So what's the difference? Well, the other computational models we've talked about so far, usually we will think of some really complex components. So for example, when I talk about decision trees, I didn't mention that we can combine multiple decision trees into something called a random forest. In that case, what do we have? We have a few models, but each model is super complex. Another example is you can think about nowadays the structure of computers. Each processor of a computer is very, very powerful. And when we want to perform really complex computation, we will take a few processors and put them together and try to make them work in parallel. So that's the kind of computational models we've been talking about so far. But the model of the human brain is very different because the human brain has a lot of components, but each component, each component is a neuron is very, very simple. So the difference is having a large number of simple components rather than having a small number of complex components. There are pros and cons of having each and each can do different things, but it's interesting to realize that we are pursuing a somewhat different computational model. Let me summarize. After watching this video, you should be able to do the following. Explain why there's a huge hype on deep learning right now. Explain why we want to use a neural network to solve some problems. And finally, you should be able to describe the structure of a neuron in the human brain. That's everything for this video. I will see you in the next one. Bye for now.