 I'm interested in learning more about machine learning and my first foray into that was through a guide I found online called Machine Learning for Artists and there's a whole bunch of different tutorials on machine learning here but one that I found that I was interested in is called Picks to Picks. Picks to Picks you may have seen through this image to image demo that was circulating around on the internet where you draw a picture of a cat and it tries to interpret your image as a cat. In this guide you can see kind of how they're taking an input photo and a target photo and then they're able to colorize these images in the style of the source photo and there's a lot of other things you can do with that, satellite photos from Google map tiles. So I just wanted to see if I could do something similar and this is more a tutorial on the process of getting images into the format you need them to be for Picks to Picks than a tutorial on machine learning itself because one of the things that this tutorial doesn't cover is how to get your own set of images and if you want a really well trained model you need a lot of images. So this is what I decided to do I found this quick draw database from Google where they ask people to draw a picture of various things like planes or ambulances or angels etc. They have over 50 million drawings so there's a huge data set and it's publicly available on GCP, find links to it. I go to the quick draw data set full simplified and I was searching for fish and so you can see this fish.ndjson file shows up and it's 50 megabytes. Now the important thing to know is this is JSON data it's not images so what you get is a huge JSON file with all your data. So this is what that JSON file looks like and you can see here in this drawing key it's just a gigantic array of coordinates and that is the coordinates every time someone put their pen down drew a line and then picked it up so you could actually recreate the drawing of each person's fish but we don't need that we just need the images. So the next thing that I found was this command line interface from Mike Bostock that allows you to convert Andy JSON to regular JSON and the reason we want to do that is so that we can try to reconstruct these images in a format that we can save out. Someone else made a really great tool which allows you to view these this quick draw data as an SVG and if I were to copy this code and swap out this JSON file for whatever JSON file I had created with the Andy JSON command line interface I can view my own set of drawings and then in order to download those drawings I could just copy this SVG save it in a file save you know faces dot SVG and then I could open up that file in a program like illustrator and see all of my faces. These are the fish that I downloaded and you can see they're all kind of roughly aligned the trick here one they come in pretty small so I wanted to get them to a size that was about 256 by 256 which is the size that Pix2Pix needs. I wanted to make sure that my grid was of a size that allowed all of these images to be self-contained what we don't want is anyone particular drawing you know sort of crossing over a grid line because then what we're later going to do is slice these images up programmatically so that we don't have to you know copy and paste and save out each image separately so we're going to be using image magic for that this is going to tell me okay I need to make slices of images that are 350 by 350 pixels. Once you have that run image magic and so I've saved this out as fish.gif and if you have image magic installed you can run this command and I'll post all this code but to convert this one image crop in a 30 by 19 grid so there's 30 columns in this image and 19 rows yours may differ but that's what I set up so 30 by 19 repage a join and save them out into a directory and you can call this whatever but I called it the tile directory and then this is your enumerator so one two three four five six etc so once that's done you'll have a directory here called Tiled and it will contain all your images after you've got all your tiled images what you want to do is resize them to 256 by 256 so this is the code in image magic to do that we're just running a for loop and we're saying for all the images in Tiled directory that end in .gif convert them resize to 256 by 256 and keep their original name and extension and we're putting those in the size directory so here's my sized directory and you can see I've got all these fish images all right so now that we have a fish we need to find photos of fish that we want to copy the style of and train our model on I found this in internet archive book image library and it has all kinds of really cool illustrations it'd be great to be able to just bulk download all of these and so I found a chrome extension there's a lot of them out there but this was actually the only one I could get to recognize images from Flickr by Vlad Sadev so I downloaded extension and if I were to use it on this page you can see it recognizes all of these photos and then you just select the ones that you want and click download so you can see here this is what I downloaded as far as fish and they're all different sized images they're all 640 pixels wide those but what we need to do is get those to be square and eventually down to 256 by 256 pixels here I've got some instructions for how to size those images appropriately again we're going to be using image magic first thing that we want to do is sample the background color and the way we're going to do that is just sample the very first pixel and resize the canvas and fill the new space with that color so that's done with this command for saying this is our image in color space RGB format pixel and then we say convert background equals that color this gravity center means we're going to be resizing from the center so if we had say a 256 pixel image it would expand outward from the center and fill that new area with our background color so because these were 640 pixel images we use this extent 640 by 640 and we're just giving a file name of fish dash T is the the iterator so one two three four five six dot extension and in this case that extension is JPEG so once we've got these to be square we want to size them down to 256 by 256 so that's that process now the next thing we need to do is we need to get pics to pics to train itself on these photos and we do that by combining our drawing with our photo and we're going to use it I'm going to just do a hundred of these images and so my command to do that is for I in one to a hundred do montage and then we're going to take images from the size directory so fish dash I that gift and then from the size directory of just our regular fish so we're combining these drawings with these photos we're tiling them two columns by one row and the geometry is 512 by 256 or essentially two times 256 pixels and then we're going to save it into the montage directory and call it fish dash I that's what I end up with what pics to pics is going to try and do is it's going to train itself to say that this drawing represents this fish this drawing represents this fish etc and we're going to define a number of epochs or iterations to run the model through for each image so once we've downloaded pics to pics we just go into the directory containing this pics to pics dot py file mode train in pic directory is our montage directory output directory is our model directory which direction a to b means that we're training a images to become b if our direction was b to a we would be training these photos to become these drawings and these are those epochs or epochs I'm not sure how to say it we're going to train each image through the model a hundred times it's not really enough to get good results in our image set of a hundred images is not nearly big enough to get great results but this does take a while even even at a hundred epochs with a hundred images it took me about 30 minutes to train my MacBook Pro and it's it's really processor intensive so once we have that we end up with this model directory and it's just got all the code that we need in order to create our training set here's all the data about 680 megabytes now we need to use pics to pics to run a new set of images through that model and so in order to do that we need to create a training set and the training set will essentially be new drawings paired with a blank image on the right so we need to do montage again to combine some of these drawings with just an empty white square and so the way that we do that is we run another loop in my case I'm just saying for I in 101 to 200 so we say convert them fish 101 102 etc. gift our gravity is going to be coming from the west so we want our image anchored on the left side and the white space to be on the right background white type true color that makes sure that when we convert these to JPEG they're not in a grayscale color mode because see the photos have an RGB value but these gifts are just black and white so this true color converts them to an RGB color space essentially extent is again 512 by 256 and we'll put them in the training directory so once we do that here's our training set and now we can pass that through our models we go back to our directory containing this pics to pics dot py mode is test input directory train output directory I'm going to say is Jen and the checkpoint is our model directory so that's just going to pass everything through the model and once we do that we get this Shen folder and it creates an HTML file and all of our images and if we open that HTML file in our browser you get something like this I actually kind of switch some things around so that you could see each phase side by side these samples are not the ones that model was trained on because it it runs through those epochs so it trains itself on all sorts of combinations of these images and these photos this is just kind of a way to see things side by side and you can see our output we get some pretty crazy looking fish again it's not going to be perfect photo realistic rendering but that's kind of what I'm interested in I'm interested in the weird things that come about through machine learning trying to interpret our photos and images in our world anyway I'll post this tutorial online and include a link with the video so that you can copy all the code and do this yourself and download your own image sets and convert them to the right formats thanks for watching