 So hello everyone we want to we need to start Okay, because we have only Few minutes okay to talk about a lot of things Welcome to our presentation about the video data and open source intelligence My name is Frank Gomez. This is this are Jimenez We work at Divo, okay Today we went we were going to talk about Video analysis and phase recognition. Okay, and the idea is we want to start just making a demo Okay before to explain how to do all the things that we are seeing in the in this demo. Okay, so Like you prefer Okay, so We need this mobile. Okay, this mobile is just an Android and have a steaming server so the idea is We we want to Share our mobile phone with you to try to recognize The guys that are here tonight today. Okay No Okay, we are getting the the streaming. Okay, perfect. It's working. So I need a volunteer Okay, these guys are the perfect guys to a star you can Okay. Oh the system is Recognize you and you okay. These guys are working on Divo. Yeah, obviously You want to try Nothing unknown Okay, the the next minutes we will try to do this but using or LinkedIn contact to search if any of you are in or LinkedIn contacts so Wait wait We are talking about the video analysis the idea is Today, we're trying to explain you the importance of the processing of the video right now because it's one of the things that Government and companies are doing to get a lot of information. Okay First we will start talking about the open source intelligent. How many of you know? What is open source intelligence? Nobody To guys. Okay, the open source intelligence is just to get all the information that you can Get from different sources. Okay, mainly I don't know governmental information Network social networks anything else. Okay Open source intelligence was defined in 1992 1992 with an all-style definition, okay, but since soul seems out of date, but the definition is Good for our time or nowadays because the only the The format and only the way you are getting the information is changed. Yes, but talking about newspaper and the cities The city is necessary. Okay a modern definition is this okay open source intelligence Is the data collected from policy available sources to be used to be in an intelligent to be used in an intelligent context? Okay, so the idea is get a lot of information and Apply this in a different use case. For example, national security counterterrorism cybertracking and monitoring people and even to Get calculate the Roy in your advertisement and so on So today we are focused on the video analysis. Okay, because by 20 by Twenty twenty-one twenty-twenty-one 80% of the world's internet traffic will be video. Okay video is one of the things that is growing exponentially in the social networks. Okay, so Nowadays in this year 90 million of terabyte per month per month and on internet are video. Okay, and only two percent are from Surviolence so Video is a really great opportunity to get a lot of information to use in your companies or in the governments So you can see all of the networks support to upload video to the to this networks Next we will try to explain quickly a video that that is just processing a Easy propaganda video and the content is maybe some sensitives. Okay. We apologize for them a month so the idea is You can get a video. This is typically from the military intelligence or No mass. It's more to the Cia This kind of hatching. Yes. No, no, no the typical information that you are looking for in a video. Okay, the idea is With the information about the context of the video the building the buses you can get where the the video was recorded, okay this case just using Google Maps to Correlate the information that you are seeing in the video with the information that you can get from Google Maps and then Even you can get you can know when the video was recorded because in this case This guy's it was killed by this terrorist and the the blood in the in the ground and the brushes You can show when the the the blood appears in the Google Maps history Screenshot, okay, so the information in this video is not only a guy is killing other guys Okay, there are more information in the video, but today we are focused on the Face recognition, okay, in this case because the face recognition is an application which has a lot of advance advances in recent years, okay before this early start Explaining you in detail how we can Correlate how we can detect the faces and compare with the faces that you have in your database I want to explain you in a six simple steps Okay, first you need to know where the faces Using hug or CNN you can get where the faces in the image, okay Then you get get the same you can get all the information from the face in this case Creating or calculating the landmarks and then you can scale or rotate or cutting the the face that you are getting Because you need to put in the same position that all the other face that you have to do that in your database When you have this you can calculate the vector array that define this face. Okay in this case there are 208 Measurements that define this the face this face, okay, and then you can compare with all the face that you have in your database this is a typical Correlation about the face that you are getting from a video and feel that you are in your you have in your database, okay, so Okay, what are the faces? I don't know try to find Okay In the first step we are talking about where are the face the first problem in the first step is identify where are the face and The recognition of space is a very complicated task in some case like you can see in this image that will further and Charles Smith are it's like a twins and Not only face Recogniting objects in some case is a very very complicated task No, if you can to try to identify Chihuahua's and muffins in this image is very very difficult this kind of things Make The the new algorithms not new but is with more advanced in the last year And the first of the techniques that see is hook histogram of oriented algorithms the hook techniques was developed in in 2005 and Try to find in In one image a pattern the pattern is the movement to the more darkness points in the image and with this move meant We make or we draw a vector in this direction in this image or faces You can see the hook Result in the little window and in this hook result you can identify the face patterns and The the technique is to compare all the objects all the cells in the image with this pattern and When you find the same pattern you find a face Okay, let the classification for this kind of Patterns is across a support vector machine support vector machine is a All algorithm in machine learning, but it's very useful in this case because you can and You can separate the plane in two areas and in each area you can find Faces or not faces in the image the separation is between a Hyperplane in n dimensions you can see in two dimension and is aligned but in n dimension This is a hyperplane as true fights in the in the in the scope of the of example Sorry Well, the next method to find to try to find faces in in image are convolutional neuron network You can see in this image the transformation across the convolution subsampling layers, but The first of this step is a convolutional Liar the convolutional liar is very easy to understand because I have a matrix the convolutional matrix So there are the kernel the matrix in the middle of the image I think this kernel you can put some values in this case This kernel is used to detect vertical edge in a image you can an image in the in the left part of the Presentation and you can move the kernel across all the image and Make this this operation that you can see in the first steps seven Times one plus two times zero and their final result go to the final convolution matrix For example, if you can if you need to detect vertical edge in an image You can see the image in the left hat in the left side and with this kernel You can detect in the center the vertical the vertical edge You can use this kind of convolutional matrix in GIMP GIMP have the in tool the option to apply this kind of matrix and You can practice and try with this kind of convolution Well, the second step in a neural in a convolutional neural network It's a sub sampling the most used operation is max pooling in this case. It's very easy You could the matrix in Portions in this example two times two and In each portion of the matrix you can choose only the highest value in this case for the first two times two Portion you can choose the seven in the next the nine and this kind of things well these operations are Previous a neural network and You can concatenate a lot of these kind of operations to obtain in the last step of these layers of a vector flattening all the Previous matrix and is this vector is the input to a normal neural network with her inputs Her layers her Outputs and her way it's and like a normal neural network okay caution You can use CNN in a singing in video because the wall can be explode by recursions Okay, when you choose to To identify where are the face? her or CNN well her has best accuracy Pardon her Has faster and can be used in video in real time and CNN have best accuracy And can be used in processing offline videos and these kind of tasks Okay with these techniques we only we've not only Recognize face can be used to recognize objects like flags in this example The next step We know the coordinates of the face in an image the next phase is Calculate the face lag mar the face lag mar are only points in the nose in the ace in the a brown to identify the face And this kind of points can be a lignet to the same position For the next step. This is a best technique to the to obtain better results in the next phase only apply apply a fine transformation like rotation translation and scratching and this kind of things and You obtain all the landmarks in the same position a league to the same position Well, the the last step in face recognition is a face encoding face encoding is used an algorithm creating to 2015 by Google and Obtained 128 measurements of each face. This is like a automobile plate or plate of The person because this number is the same or very similar for the same person But are very different for persons that are not the same person This is like a vector in a 128 dimensions when you when you try to identify the faces you only compare these two vectors and You calculate the Euclidean distance between these vectors and try to find if a person is The person that I can find In this case you can obtain true positive the person is and is recognized False positive the person is not the fine person, but is identified like a person That I try to find The false negative the person is but Don't identify the person and the true negative the person is not the person that I can Find and is not recognized like this We we use a tolerance to move between this balance between true positive false positive and Using this balance you can move the tolerance Move the rock to obtain the result that you that you can try Okay Yeah, if you like We have time to try the another demo using this The mobile okay, and we will try to detect who of you are in our LinkedIn contacts So maybe can I use the camera? Mm-hmm Tango Okay, you game looks like Samuel is contact my LinkedIn account you you know me No This is a typical false positive. Okay. Yes because in Facebook Sorry in LinkedIn. We have only one you want to photo Person this kind of training with With a few photos of the same person Launch a lot of false positives. I know all of you You are in my LinkedIn contact No And this is our contact I Don't know but I think no Okay, the problem here is you need to to to take the the video in a good Light okay a great camera right now we are just sending the the video in streaming today to the laptop but The idea is if you only have one picture from your target you can get the target now in a good way Okay, but we have here a video That we are using the the same approach but with Or the guys in the in the company I think Okay, here we are trying to get from all the guys in the in the office who are in the LinkedIn contact from Cesar My LinkedIn contact, okay The idea is here The the the training is the same we have only one picture, but the camera is good and the light is good Here we can show Some of guys are in the both of Contact, okay So this is the the the idea is you are getting all the information So you are getting the information from the face and you are trying to to recognize in base of your database but now We're going to show the next step the next step is this unknown unknowns. How many of you knows where this expression come from? The unknown unknowns nobody Okay, don't I'll explain us There are reports that there is no evidence of a direct link between bag dad and some of these terrorist organizations There are known knowns. There are things we know we know We also know there are known unknowns. That is to say we know there's some things we do not know But they're also unknown unknowns the ones we don't know we don't know So that is the unknown unknowns. Okay all the things that you know that you know That you don't know. Okay. So this is to introduce our use case Our use case is just to try and to process all the information from isis propaganda I don't know if you are familiar with this, but isis is the the stomach islamic state Okay, and it's polishing in social networks a lot of propaganda Related with the with the terrorist. Okay So the problem is the social network can't stop This kind of videos and we think we need to to take advance from the videos and the idea is We are getting the videos from the propaganda To classify all the people that is in these videos. Okay, because the guys you you know The people that are in the database from Interpol fbi. I know It's easy to to to to the tech in the in a video to to know that this video contains this Okay, the problem is we need to Catalogate all the people that is in this kind of videos. Okay, because it's the first step to identify these guys so Our demo is it's completely live. Okay. We have here On an storm. Okay We have a topology The idea is we start the topology right now using Two videos just to show how the topology is working And then we try to explain the architecture of the topologies. What is the topology doing? Yes All the code was published will be published in our Github account of the company github account as you can obtain this kind of proof of context in four weeks Or not Sorry You in this in this proof of context you can try to use with videos photos and and whatever you want and the idea is all the They the ideas in the presentation Can be used for us in in your in your computer Okay, we start the topology the topology right now is just waiting for a for a video in a in a folder Okay, the idea is just to get in this this video using Different queries from the from the group from youtube. Okay Okay And then Start working Okay, the results are in the phase folder and we are getting Two two kind of results the the guys that we know Okay, the guys that we have in our database we train the database with these guys And in the other side, we have all the people that we don't know Okay, but we don't get only the people that we don't know we don't get the people that we don't know in in different folders and Each folder is the one folder to one guy. Okay because it's it is We are trying to to get all the all the people in the in the video To process in a bad processing just to identify using another techniques. Okay so because this is a hard processing and Maybe we need to to wait To see some results here. We have the first guy zero folder the second guy. Okay Maybe we can put this Running and then continue with the presentation Okay Maybe you are like Homer Simpson right now Now we're going to run a few tests. This is a simple lie detector I'll ask you a few yes or no questions and you just answer truthfully. Do you understand? Yes Okay, so Let's just start We need to explain the the architecture architecture is quite simple We are using open cv just to to processing all the image. Okay And then we are using a batch storm just to to the to the real time processing because we are using delete Join to Faith recognition library in python that help us to to processing all the information in the in the image Okay, the idea is we are we are just getting the information and trying to to to processing at real time Okay, this is why we are using the storm and not only just an an script. Okay So the architecture is so easy topology is getting the the video and is Getting some frames from the video because we are not processing all the streaming We are just processing the frames from the video and sampling this This video then we are Identify all the face in the video Okay, and each Face is going to the faster recognition. Okay, the faster recognition is just to query or database To know if this face Are isn't your in or database or just putting in a The sorry and put in the know people. Okay, if you don't know Who is this guy you put in the next step and the next step is trying to detect The relationship between the unknowns. Okay, we are putting the Guys that we don't know in the unknown people And then we try to process these unknown people to Correlate to put join all together in different folders. Okay, the the question is What kind of algorithms we are using to try and to catalog all the people that we don't know Why we are using Chinese whisper. Okay I'm talking about this kind of algorithms Well, the question that we need to resolve is In this video How many different peoples appears in this other video? How many people appears that appears in the first video? This kind of task needs to agroupate or or make a group And with all the face that you can find in a video Okay, for this we are trying some Techniques the first of all is a proximity has a proximity has Is a hash function but not like a cryptographic function because in this case can be used over image and this function Take an output E and the output is Very similar when the image is very similar This kind of techniques don't work properly because The background of the image the lighting in the image can can modify The output of this proximity has and don't work properly the second And the second technique or algorithm that try is Cummins algorithm is a very use useful algorithm When you try to classify it without training and in this algorithm you Choose the groups of the of the of the data in this case the group of the face Basset in a partition of the euclidean space And choosing a centroid around all the similar Objects or the similar data are agrouped in this case the the algorithms work properly When you know how many people are in the video how many different people are in the video But in this case don't work properly because the estimation of the number of centroids No We don't we don't we don't obtain a good result with this kind of estimation Well What algorithm of techniques works for us the chinese whisper The chinese whisper is an algorithm that used in natural language processing And it's very simple to understand and very simple to implement because only Follow these three steps Organize all the people in your case in a groups Linking this group for the proximity of the 128 measurements that are staying the Across the recognition phase and Is work properly and group every people in in in a very simple and speedy speedy Okay Go back to the to the demo I don't know if the Okay here for example This is a guy that we have in the database in this case is Abulafa is all Okay, we are training the We are half we are in the we have in the database So we are detecting this guy in the in the video, okay And in the other side, we have these guys the the groups in the whisper algorithms that are The same person the problem is the The algorithm works properly when the time pass, okay You need a lot of time to move The guys in the different groups and put in the correct group. Okay, because The the right now the algorithm is just trying to move trying to to to overpay this kind of persons Okay, yes, because it's working in real time And in the algorithm if you see the step previously in the in the slide the algorithm Moves the person from one group to other group around the time until the The the works converge or predeterminate number of steps But in this case The algorithm is working Until all the groups converge and have a good approximation This is a phase positive because all the phase that we have for this guy the abu Alback daddy are just in front. Okay. It's not the the side of the of the face And in this case this guy is Just trying to the the algorithm is trying to identify a phase, but the phase is not in the correct position. Okay So just Yeah, as you can see the false positive is a very complicated problem to solve in this kind of work And in all the work that you can see in presentation only So the true positive case in this case We prefer to show what is the reality of this kind of algorithms work properly, but need a lot of training with correct phases and Required to choose the tolerance For that kind of detection that you can obtain So what what is the social impact right now with the face is the phase recognition? Okay, the the phase recognition is just in the in the companies is just in the In your life. Okay Companies and government are trying to to know where are you to know what what are you doing here? Who are you? Okay, so the retail stores are using the face recognition to know how many times you are going to the To the to their store. Okay Maybe they don't know your name, but they know that you are there and you are buying something. Yeah in this article Talk about Walmart Tesla Tesla is deployed a massive new autopilot neural. Okay, what means this? Tesla has a lot of video where people With people with cars with another with all the streets. Okay This information right now is just using to to the cars, but This information will be sell in a moment. Okay, and this information is a great information to know Where a guy are in a moment. What are they are doing? so Even the the visual search is one thing that the the companies are doing. Okay, you just See, uh, something just search and buy. Yes. This is a demo in your snapchat Visual find So even you can buy A webcam including all these features. Okay all the features in face recognition are included in the amazon In this amazon webcam, okay And then the the the best example of this is the china social score program Because they are trying to to Correlate all the information of the date to they want to know all about the citizen just to to Put an score in each citizen and then because I don't know maybe punis the the guys Without fast trains without the Good schools for hotels Bum for flies. So we need to defense. Yeah, okay. It's time to defense What can we do to defense against this? You can spend money and get this cup. Okay, this cup is just trying to To to blind to blind the the cameras. Okay, but you need to to put all the day the The cup and you need to pay. Yes, this cup works properly with the night vision cameras because the leds are infrareds Okay, in another way you can just buy Some glasses It's better but are cheaper. Okay, or even if you don't want to spend nothing in in this you can Yes, this is the sign of the fence. Okay, it's nice little cyberpunk Maybe you can you can do so. Thank you very much and this is our approach of the defense to the face recognition Yes, we have a defense And you got put Thank you guys. Thank you