 Hello everyone, this is AliceGal. In this video, I am going to discuss the perceptron, which is a single-layer feedforward neural network. First, let me discuss two types of neural networks. Feedforward neural network and recurrent neural network. I've drawn a two-layer feedforward network. It's a directed acyclic graph. The key property is that the network has no loops. Think about the numbers flowing through the network. They come in as input values from the left, get transformed through the edges and the nodes, and finally go out as output values on the right. These values only flow in one direction. They never go backwards. In the feedforward network, the outputs are a function of its inputs. If you know the inputs, you can determine outputs. On the other hand, a recurrent network can have loops. Here's an example. We've taken an output from a node and feed the value back into the node as an input. What is the advantage of having a loop? You may have learned this in a circuit design course. Loops in a circuit can give it memory. The circuit can remember some information while it's doing computations. Similarly, a recurrent network has memory. The complication is that the output value is no longer a function of its inputs. The output may also depend on what the network remembers about the historical inputs. Historical inputs determine the current state which influences the output. Arguably, a recurrent network is a better model of the human brain because we have memory. Unfortunately, because of the loops, it's also more difficult to train a recurrent network and to interpret their meanings. In this course, we will focus on feedforward networks. If you are interested in learning more about other types of neural networks, I'd recommend taking CS480 on Introduction to Machine Learning and CS479 on neural networks. Let's look at a perceptron. A perceptron is a single layer feedforward neural network. In this picture, it looks like there are two layers, the input layer and the output layer. However, the input layer does not have neurons. It simply contains the input values. The output layer has two neurons. Each neuron takes the input values, calculates a weighted sum, and feeds the result through some activation function to produce the output values. A perceptron can represent simple logical functions, such as and or and not. This was a big deal when it was discovered. At the time, much of AI was focusing on developing systems to perform formal logical reasoning. Representing logical functions is the first step towards this goal. Many people thought that perceptrons would eventually allow us to develop a powerful logical deduction system, which is capable of solving any problem in the world. I just told you that a perceptron can represent simple logical functions. How does this really work? Before we look at the mathematics, let's take another look at the perceptron on the previous slide. This network appears to have two outputs, but the two outputs are independent. We can split up the network into two independent networks, each containing one perceptron, and we can learn the ways in each network separately. Because of this, I'm going to focus on discussing a single perceptron at a time. Back to our question. How can we use a perceptron to represent a logical function? Here's the first question to think about. We have a perceptron with some weights on the edges. What logical functions does this perceptron represent? Pause the video and solve this yourself. Then, keep watching for the answer. The correct answer is D. This perceptron is computing a negation of the or function. The easiest way to derive this answer is to draw a truth table. With two inputs, there should be four rows in a table. Here's a complete truth table. Next, we need to stare at the truth table and figure out the function that O1 is representing. It looks like a negated version of the or function. The or function would be true if at least one of x1 and x2 is true. And this looks like the opposite of that. I found this example quite interesting. Although we have real numbers on the edges, we can use them to compute binary outputs. Looking at this example, you might be wondering, how did I come up with these weights in the first place? Let's explore this in the next question. Here's question number two. We have a perceptron with a step function as the activation function. What should the three weights be such that this perceptron represents the end function? Pause the video and solve this yourself. Then keep watching for the answer. There are many possible correct answers. Here's one of them. The key insight is that a perceptron is a linear classifier. Given the data points, we need to find the line that separates the positive examples from the negative examples. Then we can derive an inequality that selects a side with the positive examples. This inequality will tell us the weights in the perceptron. Please watch a separate video for detailed explanations. Instead of deriving the weights mathematically, we can also learn the weights in the perceptron using an algorithm. For example, using an optimization algorithm called gradient descent. I'm not going to discuss the algorithm right now. Instead, I would discuss a more general algorithm of learning weights in the multi-layer neural network using gradient descent. Once you know the general algorithm, you can use it to derive the simpler version for a perceptron. That's everything on introduction to perceptrons. Let me summarize. After watching this video, you should be able to do the following. Distinguish feedforward and recurrent neural networks. Describe components of a perceptron. Given a perceptron with weights, determine what logical function the perceptron represents. Given a logical function, mathematically derive the weights such that the perceptron represents the given logical function. Thank you very much for watching. I will see you in the next video. Bye for now.