 Here's a black and white image that we're in the middle of rendering. Let's try to guess what color the next pixel is going to be. It looks like we're in a white area right now, so let's guess white. Hmm, now it seems like we're going into a black region, so let's guess this next one to be black. Now we are in that black region, so let's go black again. If we could automatically make good guesses like this, we could use this to compress image files better. Instead of needing to store the color at a pixel, we could just tell image decoders to guess it. It is a usually decoded top to bottom, left to right, so we can't use future pixels below or to the right of the current one when the current pixel is being decoded. And to make decoding fairly fast, we'll only use three pixels around the current pixel in the guessing system we're going to develop. Okay, so when all the pixels are the same color, the next pixel is probably going to be the same. We don't have any evidence to suggest that it would be a different color. Here it looks like we're at a point in the image where we're transitioning from one color to a different color. There's one color on the left and a different color on the right. Let's extend the line on the right to match the other side. We can also extend horizontal lines in the same way. In these ones, it looks like we're transitioning from one color in the top left to another one in the rest of the area. Let's use that knowledge to fill in the remaining pixel with that color. Now we've got all eight possible inputs to our guessing function. Now conceptually, for all of these cases, we're looking at the direction of flow of the colors in the image, or the gradient. In these cases, the gradient is a horizontal line, and in these ones, it's a vertical line. For the corner ones, the color changes as we go from the top left to the bottom right. Continuing the pattern, it seems like the remaining value should be even more black or white than as possible, but we're limited to black and white, so we just clamp the values to be black or white. And finally, for the trivial cases where all observed inputs are the same, there isn't any gradient. These heuristics for guessing the colors of pixels were originally invented by Alan Pate in 1991, and this procedure is called Pate Production. Here's what the Pate Production for each pixel in this image looks like. It doesn't get all of the details right, but it's still pretty good. Let's see if we can generalize this technique to grayscale images. We'll keep looking at the gradient of the colors in the input pixels to try to guess the next one. We can do this with a surprisingly simple formula. Add up the colors on the diagonal, and then subtract the upper left value. When the inputs are diagonal, this makes the output look more like the colors on the side and pushes it away from the color on the other side of the diagonal line. When the outputs are like this line, the upper left and left pixels are close to zero, leaving only the upper pixel to contribute to the result, making it close to the upper pixel and forming a line. Watching how the output changes in response to the input can help you get a feel for how this works. Next, we can extend this technique to color images by just raking up the image into its red, blue, and green components, and then doing path prediction on each color channel individually as a grayscale image. Here's what we get when path predicting every pixel in this color image. Nice! Actually, this isn't quite path prediction. Path prediction clamps the output value to the closest input value. This is because it's often the case that predicted pixels are the same as one of the input pixels. Here's what you get when doing that output clamping on that image. The most popular use of path prediction is in PNG files where the lines of an image can be stored as the difference from the path predicted value instead of the actual value, and since the error tends to be close to zero, the smaller values will result in a smaller image size when the image data is compressed later in the PNG encoding process. Path prediction is also used in the fairly new AV1 video codec, which has some of the best video compression ratios to do intraframe compression. I hope you enjoyed learning about path prediction.