 Okay, so this is a continuation of a previous tutorial, which I'll post a link to, where we used machine learning to segment foreground images, people in this case, from their backgrounds. And in this video, I'm just going to show how to use those segmentation masks in order to do automated background replacement. And we're going to be using OpenCV for this. I'm not going to cover how to install OpenCV on your system, but I'm going to cover the script that utilizes OpenCV to do the background replacement. So I will post a link to the GitHub repo that has all the files that you need, as well. And inside of that, you'll find this folder, CV. And inside that folder, there's a file called replace bg.py. And this is what's going to be using OpenCV to do that replacement. So it's got a few other dependencies. We've got matplotlib and numpy. Everything else is included with Python, this OS keyboard and sys. And then I'm just going to go through the code kind of briefly. Really all you'll need to know is how to change a few things within this script and then how to pass the right parameters for what you're trying to do. So let me just show you first what we're going to be doing. So we ended our last tutorial with this sort of JPEG images folder where we have human beings on a predictable background. So if you notice if I cycle through this, the background doesn't change, the people are changing, all sorts of different sizes. And different genders and ethnicities and poses, lighting conditions and things like that for these photos, but the background doesn't change. And then we trained a machine learning model to essentially recognize that background as the background and the people as people and ended up with these masks. So obviously you can see black would be the background and pink here would be the foreground. Now the first thing that we're going to need to do is for OpenCV is just have these images be not black and pink, but black and white. So I have a little Photoshop action that I've set up in the repo, which you can load in and you can just run this in batch on a whole folder of images. So the action is called black and white mask, bw underscore mask. And we're going to choose where the folder is with our segmentation mask. So here we are, segmentation class folder. You've got these black and pink images, so I'm going to choose that. The destination, I have set up as none. And the reason I did that is because if you load one of these, we just want to make sure that it gets saved for web and it's got its own export action already set up. So when you load this in Photoshop, you want to just kind of run through it once and make sure that you change your export path to wherever you want it to be on your hard drive. So in this case, I'm going to put it in my photo booth article, CV masks folder. So that's going to be where I want to export to and it's going to export to that file and then it's going to close it. So that's the action. So again, once you've done that from this automated batch, set deep lab. Action black and white mask, choose your source folder. Segmentation class in my case and then we've already set up where we want it to export to. So I'm just going to hit, okay. And you can see it's going to run through these and it's turning them all to black and white. And then if we go to that folder and look at our master directory, you can see it's generating them. Once that's all done, the other thing I want to do is I want to make these images sequential in numbers so that they follow a predictable order. And that'll allow the script to work. And the way the script is set up is it's looking for a nine digit number and again, I should probably make this a bit more dynamic. But basically what it does is allows you to iterate through each image and kind of visualize it separately or output it and skip to the next one, etc. So it's a nine digit number. We want to make it that. So I set up an automator action. It's in this part of the repo called Utilities. Make nine digits sequential. There's lots of ways to do this. If you don't have Mac, you can't run an automator action, but you can do it with ZSH. You can do it with a Python script. You can do it with a bash script. I can maybe post some links to other methods of making this happen. But I'm just going to load up this action in Automator. And what it does is it selects, it gets whatever finder item you have selected. It loads all the contents of that folder, and then it gives them this nine digit sequential number. So in this case, I'm going to go ahead to my folder. What if we have the article, CV masks, and then I'm just going to run this. And see, it did that really quickly. Now they're all sequential nine digit numbers. Great. So now we've got our masks. We need to also just put our input images in place. And so I'm just going to actually copy these from my JPEG images folder where they ended up when we did our machine learning training into my input folder. And you can see they also need to be renamed. So I'm just going to run this action. And you just want to make sure those match up, obviously, that this image corresponds to this mask. And it does, should be good there. So we've got our input, we've got our masks, and then we've got our BG. And this is going to be the background image that we want to put behind our input. So behind our people here, so the one thing we want to swap out. So I'm going to put them in space. I've got my background set up to be the same size as all of my input images, my masks, so 1280 by 960, and it will do some resizing. I think there might be some some weirdness that you could encounter if you have things that are two differently sized. But best bet is to kind of make sure all those things are the same size. With all of that in place, we can now use this script. So I'm going to go in here and the main thing you want to do is, if you've got things stored in a different directory structure or whatever you want to change it to be whatever you have set up. But from relative to where the script lives, replace bg.py. I've got a bg folder, an input folder, a mask folder, and output folder. And so those are specified here. If yours isn't like that, you want to change it to be where your files are. The other thing is, mine are all JPEG images. If your foreground files are PNGs, you'd want to change this to .png. If your background image was a PNG or a GIF or whatever, you'd want to change that. Your masks, here's the extension for that. And your output files, if they're going to be something other than JPEG, you want to change that here. Everything else should be all set up. So I'll run the script and you can see what it does and then we can kind of go through the code a little bit in more detail so you understand that. But so I just want to make sure that I'm in the right directory here. I'm in this cv folder and there's my script, there's my folders. So we're good to go. Okay, if I just run Python replace bg.py, what it's going to do is going to load up the first image and visualize it and do that background swap. And there's some keyboard commands that we need to know. The first are X and Z. So if I wanted to, for some reason, this did a pretty good job of isolating the subject. But if for some reason it was too big, and I'll see if I can find one, we could expand our mask. So if I hit X, it's going to expand the mask and sort of show more of the original background of the photos. You can see we're starting to get this little black line around this person. And I've only got it set up to expand so far. So you won't be able to see it that well. But anyway, you can see what that does there. Now, if I were to hit Z, it will contract the mask and we can go back the other way. And so you could actually, if there was too much background showing, you could kind of trim that down and you can see it's tightening up around the image. All right, now, if I hit I, it's going to go to the next image in the sequence. So if I go back to my directory structure here, my inputs, this was the first one, there's the second one, and it's down the background. And we can run all of the same commands as we did there. Here, if I hit Q, I can exit out of this window. And I would have to run this script again to load up my images. If I hit S, it's going to save that active images to our output folder. So if I go to output, I can see I've got the saved 1280 by 960 image with the substituted background. Now, if I wanted to automate that, we can pass these flags. And the flag to just do all of your images is dash dash all. So if I run that, you can see it's going to start working through all of these files. And our folder is filling up with these images until it's done. So, well, looks like we might not have all of our masks matched up properly. Seems like we got some weird ones in here towards the end. So that's my bad, but you shouldn't have that happen. I've been kind of copying, pasting from various tutorials and files and things. So let's see, it starts with number 12 here. So I'll just delete those and let's say actually, this is a good chance to show one of the other flags, which is we just want our first 12 images because we know those were the only ones that were matching up properly. So you just do dash dash generate 12. And what it's going to do is just run the first 12 images, starting with zero to here. Now, if we wanted to just generate between a certain number and another, you could say dash dash start, and we'll say zero, dash dash end. Well, I'll just say 11. There you go, goes from the first to the 11th image. Okay, so that's all the features of the script. Now let's just quickly go through the code. I've got this function called maintain aspect ratio resize, which basically resizes your foreground images to fit to the background images. So if your background image happens to be smaller than the foreground images, it will sort of size those down to make sure that the subject fits within the frame. Obviously we talked about the directories. This is the initial threshold value. If you're finding that you're commonly having to hit z a bunch of times to lower that threshold or hit x a bunch of times to raise that threshold, you could just change this value. And it's printing into the terminal what that threshold value is. So if you find like 50 is a good baseline, then you'd want to change this to 50. This is number of inputs. I've got it set up at 40, am I using that? I might not be using it. I think it's just as long as it can find files, it's generating those images. I don't think I had 40 images in that example, but you may just want to, if you know you've got x number of images in your folder, change that number. This is just handling the renaming. So if your naming sequence followed a different sequence, like it was always image and then say 0, 1, then you could kind of do it like this. So at less than 10, your iterator would be 0 plus 1. At greater than 10, it's just going to be 11. So I'm just going to keep that as is because my sort of workflow works well to generate these nine digit numbers. So then it creates a foreground image and it creates a mask. It loads those things and makes copies of them just so it was not to delete the originals. Then it does this aspect ratio resize. And then it provides a little bit of a blur on the mask just so you don't get these super hard lines. I found that that tends to create just a little bit more seamless final image. There's not this hard line that makes it very green-screeny. And so that's just discussion blur. You can change these values. I've done as low as like seven for a harder edge. You could go higher for a softer edge. Ideally, maybe I'd put in a flag where you could dynamically change that value and really dial it in. And then these are all of our keyboard commands that handle the thresholding. And then these are the functions that generate all the images, generate a single image, or go through a range of images. So that's it. That's all there is to the CV script. Let me know if you have any questions. Let me know if I missed something in my tutorial to do this one a bit quickly to wrap it up because I've got to move on to other things. I'll post all of the links in the description on this video. Good luck.