 Enjoy. Good evening, folks. Oops. Good evening, folks. Thanks for being here. Sorry about the wait. My name is Raghav. I'm a student at the University of Illinois at Aberdeen-Champaign. And over the last summer, I've been working on a little bit of adversarial machine learning. And for those of you who don't know adversarial machine learning, it's basically you want to fool or break the machine learning algorithm. So there's a lot of hype around AI these days from machine translation. You can translate from French to English. A lot of image recognition your iPhones have, your facial recognition, lots of hype around the self-driving car, which I personally don't believe in. But it's good to live in the dream, I guess. So lots of people following that track and supposedly machine learning and AI is going to be the next big change. So machine learning is basically based on a concept that the machines think like humans. And so especially in visual computing, when you have these facial detectors and self-driving cars that so it really begs the question, how does the computer look at images? And how are they different in perception as we look at objects? So it turns out the computers look extremely differently. And so the images you see here are 32-bit float. And so it turns out that in a machine learning classifier, so first of all, neural networks, if you guys have heard about it, have recently achieved human-level performance. And they've been really successful. They've been deployed in your iPhones and stuff. But turns out you can actually introduce a perturbation. Or as you can see, a random noise out here that looks random to the human eye. But if you add that noise to your original image, can completely misclassify the output. So if you look at the school bus here, so this is one of the perceptions of machine learning, and it gets classified as school bus. Now, this is another perception of a school bus by the computer. And this is based on the direct encoding. So you're basically looking at a pixel-level mapping. And here's the second picture. The stripes one is an indirect encoding. So you basically take a pixel, and you're trying to map a pattern on the nearest neighbors. So surprisingly, with the human-level performance of these neural networks, you're able to get these perceptions. And hey, these are not school buses. So why are these algorithms being deployed in facial recognition and self-driving cars? Another example is these from top to bottom are the representations of zero. So this is what zero looks like if you visualize inside a neural network and you look at the intermediate layers. So something really sketchy is going on inside. And so there was this concept introduced by a few researchers from Google Brain. And so they took this image of a panda. And so they passed it on onto a machine learning classifier. And they got about 57% confidence that, hey, it's a panda. Then they add this visually imperceptible noise, which is imperceptible to the humans. But when they add that noise, it gets completely misclassified as a given. So to give you a perspective, if you're self-driving car tomorrow, misclassify as a green light as a stop sign or a stop sign as a green light. So that's a precarious situation. Moreover, last year, I think, a couple of researchers again from Google showed that when you take these so-called perturbed images, adversarial images, and you print them out on a sheet of paper, and you feed them as an image digitally, again, it still gets misclassified. So here you see the image. So they actually printed it out on a paper, the noise, the noisy image, and it got misclassified by the algorithm. So that shows that these perturbed images, these noisy images that can fool the classifier, can exist in real world. They also tried that with a 3D turtle. So they actually 3D printed a turtle with these noisy images. And this was able to misclassify it. So I don't know if you can see it, but it gets misclassified as a rifle over there. And that's not a rifle. But any guesses where the noise is in this image, in the turtle image? Any guess? Yes. That's the right answer. So the noise is right here. So that can that. Oh, sorry about that. But so yeah, a lot of work has been done on this. And people have gone crazy, researchers have gone crazy on this. It released about 15 new attacks, depending on optimization and gradient based. But they all share one common thing, that you try to find the internal structure of what's going on. And you try to replicate it. And then you try to fool it by going in an opposite direction. And these attacks can be carried out in two ways. One is the untargeted and the targeted. So by untargeted, it can just misclassify to anything else. Whereas targeted is the most dangerous one, because you can actually target what you want to misclassify it. So as I said, you can misclassify a stop sign to a green light. And that's a dangerous situation. Again, white box and black box, if the attacker or the adversary knows what kind of network you're using algorithm. Another interesting thing last year, so this is one of my friends from Google. And so he's working on these adversarial, noisy things, not just for images. They can actually also exist for speech. So let's look at this. I don't know if it's going to give me a. Did you guys hear? Sorry about the guys in the back. But it says, without the data set, the article is useless. And that's the original output. So this is basically a speech to text converter that's used in Siri and Google Now. And when this speech is passed on, so the output is, without the data set, the article is useless. But if they passed on a slightly noisier version of it, so it's still the same speech, but the text that they get is OK Google, browse to evil.com. So that's kind of sketchy, right? So if you take a phone and say, Siri, hey, what's the weather outside? And it takes you to a malicious site. You don't want that. So the point being that these adversarial things, they just don't exist for images. They exist for speech. They can exist for text as well. And another interesting thing is that they're not specific to neural networks. They're spread across all these networks. So why do they exist? And what's the reason why these things are happening? I mean, we're just passing on simple images and asking the algorithm to learn. So the reason is a little mathematical, but laid out as simple as possible. So a neural network or a machine learning algorithm tries to be nonlinear. So let's say a cubic equation or x to the power of 5. So a neural network is typically to the power 100 or in hundreds of dimensions, thousands of dimensions. So it's extremely nonlinear. But the nonlinearity lies in its parameters, but the input and output are still very linear. So imagine you're separating A from B, and let's say it's a linear boundary. And so the distance between A and B is still small. So you can point or find a space in the opposite class and try and misclassify it. Where are these adversarial images being used? So the captures, I'm sure most of you must have seen this. So to find out if it's a person or a robot. So now with all these state of the art neural networks and you can do image recognition with human level performance, these captures are useless. So they're replacing these images with adversarial images so that the robots cannot recognize it. Another indirect application of adversarial or these noisy images are generative adversarial networks. So it's basically a combination of two networks that combine to produce an output. So for example, this guy, the horse, I'm sorry, the horse. So they feed in two images. One is the horse, one is the zebra. And it is able to generate a probability distribution that converts it from a horse to the zebra. And you can use this for generating more images. So it's basically a computer's perception, a machine learning algorithm's perception of the images that it can generate. This can be used for more data generation for machine learning or can be used for applications like these style transfer. So interestingly, a bunch of researchers were able to transfer style from Picasso's paintings to regular images of people and kind of mix it up. But adversarial images, they're malicious, they're harmful, but they're not perfect. So interestingly, simple operations on adversarial images, if you crop the image, so let's look at the guy here. And the dog here, so the adversarial image gets misclassified as a tennis ball and look at the confidence score, 75.65. So it's still, it's pretty confident. But if you crop it or magnify this, it returns back to its original class, Labrador retriever. So that's very interesting. So that means there is existing a very specific pattern in that image that is only maintained if you do not make these input transformations. Same thing for brightness, right? So in the case here, it gets misclassified as a tennis ball again with 75% accuracy. In the second case, when I increase the brightness by 50%, it still remains tennis ball by 76%. But if you increase the brightness too much, it switches back to its original class. So the reason I'm showing you all these is that, okay, adversarial images are scary, but if they cannot withstand change of brightness, cropping, magnification, how can they exist in the real world? Because in the real world, your self-driving cars are not gonna look at your images at only one angle or only one size. So it's gonna be a variety of sizes or a variety of lighting conditions. I'm gonna skip over this. So again, coming back to the point that can adversarial images actually fool self-driving cars and detectors? Answer is no, at the moment, no. And, you know, a simple illustration here. So this is the original image, this is the adversarial image, and I'm not sure if it is large enough, but it still classifies both of them as a person. And the reason being is that when you have some, like these croppings, so for example, this bounding box is a cropping operation, so it loses its adversarial property. So at the moment, detectors and self-driving cars are not susceptible to adversarial examples. But still, your simple algorithms are still very susceptible. So for example, your facial recognition, people, the researchers were able to fool iPhone's facial recognition by models by adversarial images. So is training on these images a solution? It might be, but it's too bulky. You need to generate too many corner cases. It's like testing a code. If you're a developer, you need to find all the extreme cases. But what if you have a thousand extreme cases? So it becomes kind of cumbersome to find all those extreme cases, I guess. So yeah, adversarial images are still a very prominent problem. Yeah, it's still a very common problem. People are working on a lot of defensive strategies. And people are also working on extending these to detectors, as I showed you, for the cropping operations and stuff. That's it. Thank you. Any questions? So the way it works is, if you see the epsilon term here, which is 0.07, so that term actually determines how much you're perturbing your image. So it kind of becomes like a beam search. You wanna have a balance between how much you wanna perturb your image. So it's a strong perturbation, strong misclassification. But you also don't wanna perturb it too much. Otherwise it becomes perceptible to the human eye. So the more you increase the epsilon, the computation increases. But that's a trade-off. I, in fact, used all the possible networks. And one interesting fact is when you, so if you're familiar with most of the image recognition models, so another interesting thing about adversarial images is that they don't transfer very well. So for example, if you use a VGG-based network and a ResNet, if you generate from VGG, they don't fool the ResNet. And so there's some transferability issues out there. This is really cool library by Google called Clever Hands. It's maintained by one of their security staff. And that's really cool, that's what I used. Okay, thank you.