 So, welcome. I am Dead Program. In the real world, some people call me Ron, other couple of people call me Dad, but I am Dead Program on GitHub, Twitter, and all the other important places. But I'm not dead, not yet. And I'm the ringleader of the hybrid group. Hybrid group, or boutique software consultancy that specializes in writing software for hardware companies. So Intel, we've done a lot of work for them. Also, they are sponsoring my travel this year, so thank you so much to Intel. Really appreciate it. And a company called Sphiro on this little movie about Wars and the Stars, and it's a real robot. Anyway, that's all I can say. So, what we've done a lot of open source work, the most well known is a little project called GoBot, and that's why I was here last year speaking about GoBot. Anybody have the chance to play with GoBot yet? I love you. But I'm here today to talk about a new project called GoCV. So what is GoCV? So, GoCV lets us use OpenCV, which is the leading open source computer vision package. It literally supports hundreds of patent-free computer vision algorithms, and it lets us use OpenCV from our favorite programming language, Go. You can also use the Intel computer vision SDK, which is a special version of OpenCV that's got some hardware acceleration baked in along with a couple of other neat things. So first of all, what is computer vision? So computer vision, according to Wikipedia, is computers that can analyze visual data to perform tasks or activities normally associated with human beings. So what does that mean? Well, we can detect motion. We can recognize people or other objects. Telepresence, whether that's medical telepresence, office telepresence, or my personal favorite, telepresence on other planets. And then autonomous vehicles, trains, planes, automobiles, and submersibles, don't forget under the sea, and augmented humans. You might be thinking about people with superpowers, but maybe we could also think about doctors who are getting a chance to look at MRIs and radiographies and spot things that the normal human eye cannot. So why would we actually use Go for computer vision? Well, it's the same reasons that we'd use Go for everything else. Concurrency, portability, and performance. Okay, did you get those? Concurrency, portability, performance. All right. We'll work on chanting later. So how GoCV works is that it uses Go to talk to something called CGo, which is part of the Go tool chain that lets you compile C code right in and it wraps C using C++ because OpenCV is written in C++. That sounds kind of complicated. Luckily, we've done all the work for you so that it works really well on Linux. It works great on Mac OS and it works on Windows. Yes, I said Windows. According to Brian Kettleson, 84% of desktop users are still running Windows, so let's take care of the Windows users, people. So the whole world of video, I'm ready for my mic stand. Hi. So the whole world of video. Can we see this code? So point it towards the sound producing organ. All right. So here we see package main or usual Go package. We import the GoCV package and then our main function. First, we're going to create a new GoCV video capture device. That's how we're going to get the information from the camera. We're going to create a new video. That way we can actually see what's going on. We're going to create a new image mat, which is where we're going to put the image that we're grabbing each frame by frame from the video. And then endlessly, we're going to read the image from the camera. We're going to show that in the window and then we're going to wait for one millisecond. That way we can keep up with it. All right. So that's the simple hello world of videos. So if we go over here and we run this, hello world. It's video. My mic stand. All right. So now that we've seen that, let's get into the matrix. So what is a mat? What is a matrix? A matrix is really the fundamental data structure that you're going to use when operating with OpenCV. And so it could be a two-dimensional, three-channel array like we were using, like you expect of normal images. But the mat in image OpenCV is actually quite sophisticated. It could be used for two-dimensional, one-channel. It could be used for three-dimensional, four-channels. You know, to keep vectors lists. It could be used for quite sophisticated calculations and it's all handled by OpenCV's memory allocation for you. So you don't have to worry about that. So I'm going to show four little mini applications all written using OpenCV. So face tracking. You've seen this a million times, but you know what? Face tracking isn't cool anymore. Is it? Like, do you really want your face to be tracked? Now I'll tell you what's cool. Face blurring. That's what's cool. So, but we still have to be able to track your face. So we're going to use something that's called a cascade classifier. So it turns out that a couple of researchers, Viola and Jones, discovered that this odd characteristic of a mathematical object that's known as a Har cascade, or excuse me, a Har matrix, by using four different patterns we could analyze any image and by refining that further and further down we can recognize features of that image. Luckily you don't have to worry about that because we just have the GoCV's cascade classifier. That's going to do all the work for us. So let's take a quick look at some code. So it's still package main. We've got some of our standard Go packages. GoCV again. So we're going to parse our arguments so that we can get the right camera ID. Then we're going to open the webcam, same as we did before. This time we'll do some error checking. Also we'll close the webcam when we're done, like good citizens. When you're using Sego, you're going to need to free the memory that's been allocated using C because Go's memory management is not going to kick in and save you. So that's very important to keep in mind. So we open our display window, prepare our image matrix. We load our new cascade classifier, the one that's going to identify the faces, and then we read in the face, same as before. We run the detect multi-scale so that we can identify the face by doing that scaling operation down. And then we're going to actually blur each of the faces it finds. So we're going to use a Gaussian blur, which should make it fairly difficult to see. Then we're going to show it and then run it. So let's plug in the other camera real quick. Can I resize the code? No. Okay. Because these are the first statement that never executes. That might be not real good. Let's see if the lighting out there is better. Don't worry, you weren't there. You in the front have plausible deniability. I know you feel safer already. All right. So motion detection and tracking. So for this, we're going to use something called background subtraction. So background subtraction is a very useful technique for us to be able to separate from what's in the background and what's in the foreground. That way we can easily detect when something's moving around. So the way that the background subtraction works is it's going to use what's called a mixture of Gaussians. So real quick, Gaussians is a normalized distribution, you know, like the bell curve in school. A mixture of Gaussians is when we're actually going to look at the red, green, and blue and determine the Gaussian distribution for each of those. And then by doing a running mixture of Gaussians for each of the pixels, we can develop a heuristic whereby we can look at each pixel and determine whether or not it's part of the background or part of the foreground, even when the background changes. Luckily, you don't have to worry about all that because the GoCV background subtract or Mog2 handles it for you. So here's some code. We're going to skip parts of this because time is precious. Same sort of thing we're doing with some standard Go packages, GoCV. We're going to open the video capture device. We're going to open a window. This time we're going to have a few different maps. The one that we pull the image into, the one with the image delta, which is the foreground only, the image threshold because we have to clean it up a little bit to actually do the image tracking and then we do the background subtraction. So we grab from the webcam and then we apply in GoCV and in OpenCV, the apply function is used by all algorithms to take that algorithm, apply it to the source and result in a destination that has the results of that convolution. And then we're going to clean up the image using a combination of a threshold and dilate, which is just to get rid of all the extra little bits. We're going to find the contours of it and then hopefully we'll detect some motion. Mike stand. Mike check. Hi. Nobody said you could, nobody said you could go out of duty. I am the organizer of this room, so not only she is also the organizer. 15 minutes left. No pressure. Microphone closer to mouth. Which one though? All right. So, but there's nothing there to move. Well, luckily I have, well, let's see if we can, if I go over here, you didn't see that. All right. So let's go back over to the window and let's throw a robot in. So this is a sphero spark plus robot that I'm running using a go bot program that I wrote just because I didn't want to stick my hand in front of the camera. It seemed like, you know, not very hygienic. So it's able to detect the motion and then basically using the feature detection. It's having a little trouble because every time it hits the stand, it's moving a little tiny bit. But that just kind of gives you the idea of how it works. So we'll stop that. If you see something cool, you're going to applaud too. No, everyone's just in shock. They're like, wait, what's this guy doing? All right. So mjpeg streaming. So has anybody ever tried to just take your webcam and stream it over the internet and you discovered that it was actually incredibly difficult and you had to use like all these different tools like, you know, ffmpeg and mjpeg streaming and, you know, to pipe them together? Well, I thought that was kind of silly. Why can't we just do this in like 60 lines of go code? So we're going to bring in another package, which is the mjpeg package that we forked from a guy who disappeared off the internet. So if you show, if you see this, great work. Thank you so much. So we're going to create a video capture. We're going to create an mjpeg stream. Same sort of thing we're doing is before we open the video capture device. We open the mjpeg stream. So now we're going to start a new go team, go capture. And what this is going to do is this is going to start grabbing the frames from the video. Then we're going to use the standard go HTTP package. So we're going to handle that route by serving it up from the stream class. And then we'll just listen and serve. The capture function is quite simple. We open the new mat. We read it from the webcam. We encode it into jpeg format using one of the open CV functions. And then we just stream it out. Mike Stan. Hi. Somewhere in here, somewhere in my, somewhere in my bash history is everything I need in life. If I could just find it. I've either forgotten to install it here. Maybe where did I put it? It's probably one of these other ones. Well, let's just install it. I don't have any of that access. Oh, man. I have this in here somewhere. Well, you can do it. We don't have a lot of time. It's just all it's going to do is serve it up into your web page. I don't have internet access to actually install the one dependency. I must have switched go versions trying to try the new version of go that just got released. Sure. But it totally works and you can find the code. Anyway, moving on, tensor drone. The finale you've all been waiting for. All right. So this is a tensor flow enabled drone because if you know me, you know, I love drones and I like to fly things around plus it's go. So we prove goes air superiority. So we're going to use tensor flow and anybody here not heard of tensor flow. Yeah, didn't think so. So tensor flow is actually very interesting. Everyone thinks they don't realize it's actually flow based programming using symbolic mathematical calculations as each node. If you've ever seen something like node red, which is another flow based programming that it's very interesting paradigm. And so we're going to bring go bot into the mix again. So deep neural network is a neural network where you've got inputs, you've got outputs and then you've got hidden nodes. That's what makes it a deep neural network as opposed to just a superficial one. I guess I don't know. So we're going to use something called the inception architecture. So inception is an architecture where each one of the nodes in their own network is itself a neural network node and so on and so on kind of like the movie inception. So a team of researchers use this and trained it on a database known as image net. And so we don't have to worry about all that. Luckily, we just have the go CV net class, but we're gonna. So this code is a little bit longer because we're if you saw my talk last year at FOSDEM, this is almost the exact same drone code, except we've added the tensor flow code into it. So we are going to run the drone, we get some parameters, we control it with a joystick, hopefully. Let me plug in my receiver. Because that would be very helpful. I'm almost ready for mic stand. That's your new handle. I don't launch anything yet. Alright, so it's almost the exact same code as last time. We're gonna open the joystick, we're gonna open the drone, we're gonna also use go bots built in ability to use open CV via go CV. So the really interesting part is every time the camera gets a frame, we're going to grab the image from the frame. We're going to convert it to a 224 by 224. That's a typo in my part, blob, because that is the maximum resolution that the image net tensor flow network is going to be able to process. We're going to set the input to the neural network as that blob. We're going to run a forward pass through the network. We're going to reshape that data and then put it into order so that we're able to pull out the most likely value. And then we're going to display that in the image as I then fly around the drone. Just because I didn't want people to have to come up and put objects in front of the camera, I thought this would be just so much more interesting. So let's see what happens. I'm not saying you should cover yourself. All right, so there's you and hit that button to get rid of the extra. So now is a good time for when the whole thing's up in front of the camera to see if it can identify them. Right now it says warplane. I don't know what does that mean? Basketball? There you have it. Tentative flow powered drones. All in go. All in go. All right, so was that fun? Did you guys enjoy that? So check out GoCV.io. You can follow us on Twitter at GoCV.io, all one. Please star us on GitHub. Please follow us. Please help support the project. We've had we've got over 600 stars on GitHub. We've got 17 contributors that have popped out of the woodwork literally on all sides of the globe. At the exact same time we had contributors from China and contributors from somewhere in Europe, maybe Turkey, I don't know, somewhere in the... They were popping up so quick on Google Analytics, I literally couldn't follow them at one point. And then they all went away again. It's very sad. But so there's a lot of really exciting new opportunities for using Go in embedded systems and computer vision. And so we invite you to join us and to be a part of the next generation of Go technology. Thank you very much. We have time for some questions. But before, where is the next speaker? Cool. Okay. So please come start getting ready. And in the meanwhile we have time for some questions. Yes. Any questions about GoBot, GoCV, or whatever? Yeah. Right before your question. So GoCV is so real it has stickers. So I also bought a few GoBot stickers. So if you want stickers, come on up here and help yourself. If you have a user group, you can take a whole pack of GoCV stickers. But we're going to make you show us. Anyway, back your questions, sir. So the question was, have I done any performance testing? I have done some, not form of performance testing. And actually, so I have a hypothesis. I haven't done anything active to prove it yet, but I'm hoping to get some of the geniuses like JBD and Francesca and others who spent a long time doing performance benchmarking. So my hypothesis is that GoCV is actually faster than C++, Python, or Java by a couple of different benchmarks. It's going to be obviously be faster than Python as far as execution time. But it's actually faster than C++ at developer iteration time because if you either use the new Go 1.10 or if you just do a Go install, then it does all of the compilation for you in GoCV, so you're basically just compiling your Go code and linking to it. So it's that same speed of performance that you expect from your developer cycle using Go, but now you're linking to all the C++ codes. So my hypothesis is that it's actually superior to all other platforms, but I have no actual data yet. Any other questions? You, sir. The question is, what version of OpenCV are we targeting? So right now it's OpenCV 3.4, which is the very latest version. One of the reasons why GoCV exists, there was another Go library wrapping around OpenCV 2, but they ran into a lot of problems and developments been stalled for over a year because of SWIG. Now, in theory, I know a lot about SWIG, but in practice, apparently I know less than I thought because what we decided to do with GoCV was to actually hand write wrappers. That sounds really bad to you as the ultimate, I must have all things scripted, but allow me to interject. It takes a team of graduate students, let's just say somewhere between two and 24 months to develop a new computer vision algorithm, and it takes us maybe 30 minutes to wrap that algorithm in C and C++ and Go. So eventually we're going to have all of OpenCV wrapped and we will not have to use SWIG and we'll have an idiomatic interface that looks more like the Go code you expect as opposed to a C++ wrapper, which is a little ugly. Other questions? Yes, sir. How is the Go support for what? Ah, for TensorFlow. Interestingly, the TensorFlow support that I am using is really built into OpenCV. We're not using TensorFlow itself. OpenCV has a module called the DNN or deep network module and it has the ability to load in model files from TensorFlow, from CAFE, from Torch, and another one I can't remember right off the top of my head. I'm using TensorFlow here, but I could have just usually used CAFE, but we're not actually using the TensorFlow code base itself. We're using the support for loading its models that's built into OpenCV itself. So this is not for training. This is for taking a model that you've already trained. You know, possibly you've taken that inception model and you've basically done some retraining and some bad propagation on it to remove all the errors. And if you don't know what I'm talking about, that's okay because nobody does in machine learning. I worked in AI a bunch of years ago, so I'm supposed to say that. But we're not actually trying to train models at this point. We're just utilizing the capabilities for OpenCV to consume them. A follow-up. So the question is, have I considered training models in real time? And the answer is, oh yeah, absolutely. That's on the roadmap. That's on the roadmap. I think that Go is underrated. Go is underrated? What? Go is underrated as far as its ability to just totally take over the entire programming world. I mean, the reason why, I think Python's a really cool language. And if we look at Python, it's the only programming language that's been consistently up and to the right. You know, if you're like an investor, you're like, up and to the right? I want it to go up. If you were investing in Python, it's up and to the right. Like, you won't go wrong using Python. But it has some disadvantages that, especially as far as performance is concerned, but it's sort of the up, time's up. It's the least common denominator of machine learning, but I think we could change that if we as a community jump in. And on that, I thank you all very kindly and stick around for the next talk.