 So let's talk a little bit about typical GAN failure modes, like there exists one failure mode which is just bad images. Here you see a few of them. GANs for some reason really like eyes. I guess there's just an awful lot of eyes in photographs made by human photographers. But here you see bad bad bad images. Now what does that mean? Bad images means that the discriminator should assign very low probability to them. Now why doesn't that happen? It turns out that I think a lot of discriminators have trouble understanding the big structure of that. Like it's relatively hard for a system to state something like every animal has exactly two eyes. There are three of them. A difference between two and three. It's hard for a traditional confnet to do something like count the number of eyes. Now here's another failure mode. Now like here we see a couple of beautiful faces made by a face scan. Does anyone see something that's wrong about this? Any problems here? Well when I first saw this I couldn't do it because I'm practically face blind. But let's look at this. Here you see these two images. Look they look like they're slightly different posture and slightly different look otherwise. But otherwise this looks like it's the same person. Look at these two. Gray scale and color. But it still very much looks like it's the same person. Look at these people here. Not like slightly different posture but this may very well be the same person. And this is something and you can say this is a good way of faking images in general. Now like you don't need to be able to draw all images. You will usually get away with having a small number of them at least when it comes to humans. Now like this thing at first would completely have fooled Conrad. But we can see instead of having learned like a real generative model for all faces. It produces a relatively small number of faces. Now what's the idea of this so-called mode collapse? Now you can say there are different peaks. There's different people in our training set. And we can observe them from multiple directions. And now instead of learning a model for all of them, it learns a model that only contains some of them. And that means that under this model if we only produce green ones, like the ones, the images that we actually produce will be high priority under the model that we have. The model will just basically only understand this part. These are in the real data sets. And this is something that the discriminator won't overly be focusing. It's just like, yeah, I haven't seen those. But the fact that there's more of them here isn't overly conspicuous. Versus if it would produce an image that would be in between here. It would be very easy for the discriminator to say this should have zero probability. Now here's, for example, an example of mode collapse with something like MNEST where it might produce the same weird characters over and over. So let us get a little bit of an intuition about mode collapse. Because arguably mode collapse is the concept that really prevents these GANs from being good models for real probability distributions. So very few statisticians would recommend that you use GANs to do statistics with it. And mode collapse is arguably the reason.