 Hello everyone, this is Alice Gao. In this video, I will introduce several population-based local search algorithms, including the genetic algorithm. So far, the local search algorithms only remember a single state at each time step. The advantage is that these algorithms require very little memory. The disadvantage is that they can only keep track of one state at a time. Perhaps, if an algorithm could remember multiple states, it would have a better chance at finding the best state among them. Let's explore an idea called population-based local search algorithms, where an algorithm remembers multiple states at each time step. Our first population-based algorithm is called beam search. Beam search remembers a population of k states, where k is greater than or equal to 1. How do we update the population of k states? At each step, we will look at all the neighbors of these k states, and we will choose the best k states among all the neighbors to be the population at the next time step. The value of k affects beam search in several ways. It affects the amount of memory we need. The larger the k, the more memory we need to store the population. The value of k also controls the level of parallelism. A larger k makes the search more parallel. We will be considering the neighbors of a larger number of states at each time step. Let's take a look at a few questions. They can help you think more about the properties of beam search. What does beam search look like when k is equal to 1? How is beam search different from running greedy descent with k random restarts in parallel? Can you think of any problems with beam search? Pause the video and think about these questions for a few minutes. Then, keep watching for the answers. Let's look at the answers. Question 1. When k equals 1, beam search is essentially greedy descent. It remembers one state and moves to the best neighbor at each time step. Question 2. How is beam search different from greedy descent with k random restarts in parallel? If we run k random restarts, these random restarts are independent from each other. Each search updates its state independent of the other states. However, with beam search, we're choosing the k best states among all the neighbors of the k current states. Intuitively, the k states are not operating independently, and there's some sort of communication among them. For example, if one of the k states, called x, has some really good neighbors, then it's possible that the next population consists of only state axis neighbors. It is as if state axis waving its hand to the other states and saying that, look, I'm in a really good region, come over here and join me. Therefore, k random restarts in parallel are independent searches, whereas the k states in beam search are not independent. Question 3. Can you think of any problems with beam search? The example I gave for question 2 may give you some ideas. If one of the k states is in the region, is in the best region that we've found so far, beam search will quickly concentrate its effort in this region only. Doing this is great for optimization, but not so great for exploration. Whenever we find a good region in the search space, we tend to cluster there and ignore other parts. This causes our population to lose diversity very quickly. This problem leads to our next algorithm called stochastic beam search, which tries to improve upon beam search to maintain diversity in the population. Stochastic beam search is quite similar to beam search. It remembers a population of k states. The main difference is how it updates the k states. While both algorithms will look at all the neighbors of the k states, beam search chooses the next k states deterministically. In contrast, stochastic beam search chooses the next k states probabilistically. The probability of choosing each state is inversely proportional to the cost of the state. The lower the cost of the state, the higher the probability of choosing the state. A crucial difference is that beam search will choose the best states only, whereas stochastic beam search may choose every state with a positive probability. For stochastic beam search and the genetic algorithm, we want to convert the goodness of a state to a probability. So it is more convenient to talk about fitness rather than cost. Fitness is often the negative of cost. Minimizing cost is equivalent to maximizing fitness. If we convert each state's cost to each state's fitness, then the probability of choosing a neighbor is proportional to the state's fitness. Stochastic beam search is better than beam search at maintaining diversity in the population. Since there is a positive probability of choosing every neighbor, the search is less likely to cluster in the best region found so far. This gives us a chance to explore different parts of the search space. In a sense, stochastic beam search mimics the process of natural selection. Imagine that each state is an organism. The state's neighbors are potential offspring. The survival of the potential offspring depends on their fitness. Some will survive and others will not. The offspring that survive will make up the next population or the next generation of organisms. Intuitively, stochastic beam search resembles a sexual reproduction because each state produces its offspring without mixing with other states. Next, let's look at the genetic algorithm. The genetic algorithm draws its ideas from biology and evolution. It is similar to stochastic beam search. It maintains a population of case states and chooses the next case states using a probability distribution. However, the genetic algorithm uses a different probabilistic selection process than stochastic beam search. Intuitively, stochastic beam search resembles a sexual reproduction, whereas a genetic algorithm represents sexual reproduction. To produce each state in the next population, we start by choosing two parent states from the current population. The probability of choosing a state to be a parent is proportional to the fitness of the state. The higher the fitness of a state, the higher probability of choosing the state for reproduction. Note that a state can be chosen multiple times for reproduction. Once two parent states are chosen, they're mixed in a crossover process to produce a child state. There are many ways of performing crossover. Typically, the child will keep one portion from each parent. After crossover, the child state may mutate with a probability, mimicking a random mutation in evolution. For example, we can choose a portion of the child state and randomly change it to some other value. This final child state is one member of the new population. We will repeat these steps until we have produced k new states. These k states make up the population for the next time step. This process repeats until some stopping criteria are satisfied. One stopping criterion could be at least one state in a population has a fitness that's greater than some desired threshold value. Another stopping criterion could be a maximum number of iterations or generations. Check out the example of executing the genetic algorithm on the eight queens problem in a separate PDF file. If you have some time, check out this genetic algorithm car simulator. It uses a simple genetic algorithm to evolve random two wheeled shapes into cars over generations. It is quite entertaining to watch. I have discussed a few different local search algorithms so far. At the surface, some of them appear quite different. For example, you might think that greedy descent appears to be quite different from the genetic algorithm. But are they really that different? Let's do a quick comparison between them. The purpose of this comparison is to help you realize that these two algorithms have some highly similar components and are not that different. Recall that many local search algorithms try to trade off exploration and exploitation or exploration and optimization. Let's consider greedy descent and the genetic algorithm and think about how each performs the tradeoff. The first question, how does each algorithm explore the search space? The second question, how does each algorithm optimize the quality of the population? Pause the video and come up with your answers. Then keep watching. For exploration, a local search algorithm relies on the neighbor relation to explore the search space. However, the neighbor relation for greedy descent and the genetic algorithm look quite different. For greedy descent, the definition of the neighbor was relatively straightforward. For the genetic algorithm, the neighbor relation is a randomized process consisting of crossover and mutation. Starting with two states in the current population, the genetic algorithm produces a new state by crossover and modifies the state with a small probability using a random mutation. The goal of the randomized process is to generate a child state that's quite different from the two parents and allows the algorithm to explore a new region in the search space. Next, let's talk about optimization. Every local search algorithm tries to optimize the quality of the current state or the current population. Genetic greedy descent does this by moving to the best neighbor. What about the genetic algorithm? The genetic algorithm optimizes the population by choosing the states for reproduction based on their fitness. The higher the fitness of a state, the more likely the state is chosen for reproduction. The idea is that if the two parent states have relatively high fitness, then the child state will likely have a higher fitness as well. This scheme reminds me of a Chinese animation that I watched recently. It's called Linglong Spirit Cage Incarnation. In the dystopian world, humans were nearly wiped out. The remaining humans had to form a society with several new rules. Relationships are banned. More importantly, only people with superior genes are allowed to have babies. This idea is very similar to the genetic algorithm. Unfortunately, there are obvious ethical issues with using the scheme for the human society. That's everything on population based local search algorithms. Let me summarize. After watching this video, you should be able to do the following. Describe beam search, stochastic beam search, and the genetic algorithm. Describe the properties of each algorithm. Compare and contrast these algorithms. Explain how each algorithm trades off exploration and exploitation. Explain how the algorithms borrow ideas from evolution and biology. Thank you very much for watching. I will see you in the next video. Bye for now.