 Computer vision is a great entry point into the world of data science. I'll be demonstrating how we can use OpenShift to enable creation of the computer vision models I used in this sample footage. I'll cover a basic workflow and how the OpenShift platform provides infrastructure and tools to automate and ultimately scale the process and essentially create a factory for computer vision model training. Let's get started. For this demo, I'm using the YOLO computer vision model. It's a pre-trained model that I'm going to retrain so I don't have to build a model from scratch. As a test, I've deployed the pre-trained model to my OpenShift cluster. Let's take a quick look. It's deployed in a fast API app so I can just navigate to the swagger interface. Let's upload a simple image to test it. I'll pick an image file to upload and have our model make a prediction. Let's pick something good. Perfect. Our model has predicted 20 donuts in our image. Now let's see that prediction overlaid onto the image. And yes, we see 20 predictably delicious donuts. Great. Just for fun, let's see if it can identify something else like a helicopter. Oops. Looks like that's not in the data set for this model. So that's what I'll be showing today. How to retrain a pre-trained model on custom data. At the end of this demo, I have a model that can identify helicopters and airplanes, none of which are part of the original data and all of it on my OpenShift cluster. For this demo, I've created a basic environment with everything I need to custom train a computer vision model. The repository with instructions is linked in the description, so be sure to check that out and try it for yourself. We'll also be releasing a setup video as part of this series, so be sure to look for that. Let's start by going everything I'll be using in this demo. OpenShift is the foundation of everything used in this demo. Its ability to securely run containerized workloads makes it ideal for our purposes. Since the nature of data science is experimentation and many, many iterations, its ability to consistently deploy and maintain multiple environments and configurations makes it a great foundation for ML operations. OpenShift Pipelines is the engine that drives the operation. All the elements and tasks that make up our workflow have been automated and are managed by pipelines. As it's a Kubernetes native CI tool, it helps establish workflows that ensure consistent results with the flexibility to extend capabilities at any time. MinIOs and S3-compatible open-source object storage service I'll be using to store artifacts. YOLO is written in Python, so the workshop is mostly coded in Python. For basic model serving, I'll be using FastAPI. It's lightweight and fairly simple to deploy with the model, and it has an excellent swagger interface, which comes in handy for testing and serving models. LabelStudio is an annotation tool used to label datasets for input into AIML system training. I'll be using it to collect and label image datasets. And finally, there's DevSpaces. I've chosen DevSpaces as the primary interface for my training workflow. It's an extremely capable tool, but it also simplifies interacting with the pipeline, the model, and OpenShift itself. We start by collecting the images of objects we want our model to identify. For this use case, airplanes and helicopters. I've downloaded hundreds of images of planes and choppers. I have a copy of the zip file on my DevSpaces workspace, so let's take a look. If we look in the images folder, we see hundreds of images of fixed wing and rotor aircraft. Many different craft in many different situations. As a general rule, the more representative data you have, the better your model predictions will be. So now we have a starting dataset. Next we need to repair it for training for our model. Annotation is the process of identifying objects in your images and labeling them. This is used during model training. For this task, we'll be using LabelStudio. I've got LabelStudio deployed on my cluster, so I'll just launch the UI and open up in my project. Looking at my project settings, you can see I've created two labels for the project, fixed wing and rotor aircraft. The colors shown are what will be used when bounding boxes are around the object are displayed. I've already imported my images, and as I go through them, we see the annotations I've already created. Let's recreate one to show how it's done. I'll delete this existing one. Then I select the correct object from the labels at the bottom, in this case it's fixed wing. Then I draw a rectangle around the object. When I'm satisfied, I click update and move to the next. If multiple objects appear in the image, I would tag them as well. Since I've already annotated all the images in the dataset, I can move to the next step and export it. From LabelStudio, I just click on export. From the dialog, I select YOLO format since I'll be retraining a YOLO model. The export is saved in a zip format. I just renamed the file to something like flyingthings-yolo.zip in preparation for the next step. But first, let's take a look at what got exported. Let's go to DevSpaces to open the export zip file and take a look at what's inside. The two directories, images and labels, contain the actual images and annotations respectively. If we look at some of the labels, we see data for the class ID and coordinates for the bounding boxes I drew. We also see classes.text, which lists the classes of the dataset. My pipeline will use this during the training. Notes.json gives information from LabelStudio on the classes and environment. Now that I have an export of my dataset, I can upload it to Minio, which can then feed the entire training process. I connect to Minio and to a bucket called flyingthings. This is where everything from my model including training data will be stored. Now I just drag and drop my zip file into the new bucket and we're ready for the next step. Everything is all set for me to start custom training. The training script kicks off my pipeline and performs the entire process. Let's look at the script first. The script runs a pipeline job with these parameters, which creates and deploys a custom model. The parameters set things like which repo and branch to pull, the name we want for our model, the dataset name, and how many iterations of training to run the dataset. For now, let's just take the defaults and run it. Now that our training job is running, let's go over the pipeline tasks and get an understanding of the workflow. Fetch repo pulls our git repo code into the pipeline workspace. Git dataset pulls our zip file into the workspace and unzips it. Great class file picks up the classes.txt file and converts it to a YAML format that can be consumed by YOLO. This is a template which also identifies the folder structure for the training. If the image size flag is set, the images will be resized to a maximum width. This module can be used in the future for other image pre-processing that help improve accuracy. Distribute dataset groups the files into three groups. 70% go into training, 30% go into test, and 30% go into validation. The groups are then moved to their respective directories for training. This grouping is done randomly on each run. If the GPU flag is set, training requests a node with GPU and runs the actual training on that node. If GPU is not set, the training is done with CPUs. Once training is complete, the resulting model is exported to ONIX format for consumption by other models serving solutions. Now, all the artifacts from the training including the reports, model, class file, and exports are written to the object storage where they are tagged and propagated to appropriate folders. If the deploy flag is set, the FAST API app is deployed with the latest model. Finally, a summary of the pipeline run is presented with the parameter information. The pipeline is completed and we can see that it's deployed my custom model. Now let's use that same picture of a helicopter as before to see the result. This particular image is not a part of the dataset used for training. And... voilà! It successfully identified a helicopter from an image that was not in its original data and not part of the training dataset used to retrain the model. Now that we have a retrained model that can recognize helicopters and airplanes, let's use it on some drone footage I shot of an RC airplane. Here's the original footage. Now here's the inferencing from the new model. I've shown just a little of how this workflow, supported by OpenShift, can be used in training CV models. But let's dive a bit deeper and know how it actually enables someone who is responsible for training the models. For this use case, I'm going to use these HO scale automobiles. There are different models and colors and I want to be able to identify the type of car with my CV model. First, I begin taking pictures of all the different vehicles. I take multiple angles of each and try to be consistent with the placement of the camera. This can take some time so be patient and try to be thorough. I then create a project and label studio. I choose bounding boxes and create all the different classifications I want to use. The colors are used when the boxes are rendered over the image and are assigned randomly. You can customize them if you need to. Having defined my classes, I now import my images into the project. This is done by simply dragging and dropping them into the project. Now I annotate all my images. I select the class and draw a rectangle around the object and hit update when I'm satisfied with the selection. This is repeated many, many times. This is a good activity to distribute to a group and there are also AI integrations that can speed up object classification, but we'll save that for another video. Having completed the annotation, I export them to yellow format. Then I rename the zip file to autos01.zip. Finally, I create a bucket call autos and upload them into my bucket. Having done all that, I create a copy of the training script and modify it for this model. For this run, I'll go ahead and deploy it. Once deployed, I test it using the swagger interface. The result? A prediction that's not very confident. With such a small data set, the model is having tough time making accurate predictions, so I need more training data, but first let's take a look at the results of the training. YOLO reports on the metrics of the training which is captured in our pipeline for each training run. I can pull this from Minio and see how it ran. I'll pull it into my DevSpaces workspace and take a look. There are many useful charts, but for now we'll look at the overall results and show the accuracy of predictions over the course of the epoch runs. We see some jagged lines in our graph indicating issues with the training. With such a small data set, even 100 epochs isn't enough for a stable model. Since I need more images for training, I decided I would take more angles of each vehicle. I decided to use a simple turntable to get vehicles from multiple angles. So back to Label Studio. Import new images. Annotate the images. Export the zip file. Rename the zip file. Upload it to the bucket. Update the new data set file name. Kick off the training. And we're done. Now let's see if it's improved any. Looking at our updated results, we see much smoother transitions towards the end of the graph. The curve gradually flattens out indicating more precision as epochs increase. This pipeline saves all model iterations. So let's run both models on some video footage and see the difference. Here's the first model. We see it struggling with every single vehicle. It detects objects but incorrectly identifies the classes. The second model is not perfect, but it has significantly improved prediction accuracy. I think this simple example shows how having a predictable pipeline workflow makes the task of improving model performance much easier from the perspective of the person doing the training. I'm able to concentrate entirely on the task at hand and employ methods to improve my data set without even necessarily being aware of the underlying infrastructure. Okay, one more use case. I've done planes and automobiles so I can't skip trains. Using the exact same pipeline as the previous two use cases, I trained, pardon the pun, a model for train cars. Using the same steps as flying things, workflow, and the same image capture techniques as the automobiles, I have a model I can successfully identify the cars on a train. Just for fun, I've introduced a new car type identifying contraband or forbidden cargo. In this case, I could relay this information to a system that actually controls the track switches, and upon the receipt of an illegal cargo event, I could have it route that train to an alternate route that allows authorities to investigate. This is a very simple use case, but it shows an important aspect of operationalizing data science applications, that of integration with the real world and execution of business logic. Thanks so much for taking the time to experience this demo, and please feel free to comment and ask questions. As I mentioned, all the code and many of the assets used here are in the repository linked in the description, along with detailed instructions on setup and use. I'll also be putting out additional videos on setting up the demo and a special bonus video for building your own GPU enabled OpenShift installation in your home lab, so make sure you're not to miss that one. Well, thanks for watching, and I'll see you on the next one.