 Seems that I might begin. Hello, welcome to everyone. It's a bit weird without the public. I'll just see myself, but I'll just try it. I called my lecture today crew what is face recognition. Maybe status quo might be a more appropriate title because quite a lot of things happened in the last one or two years. So I have to concentrate a lot in what happened in the last few years. But in the last few slides we might also see in which direction this might go in the future. Let's begin a short introduction how facial recognition works in general. We have two different possibilities to take a picture. Usually you have a 2D picture, a normal photo or picture. But there's also the possibility to have 3D pictures these days. And in 2D you have the possibility to work in different white lines of the spectrum. One is the optical spectrum that we usually see. You see it in the picture on the left. We can also use near-infrared, the image in the middle. That's the light which is really close to the visible red light. It's sometimes confused with heat pictures. But this one is for infrared. And the picture that you see is the picture on the right. There are different white lines. These have different advantages and disadvantages. The visible light is quite susceptible to other light sources. Whereas the infrared has much less resolution. The second thing I talked about is 3D face recognition or 3D photography. There are different methods to do it. The first thing is shown on the left side. You project parallel lines onto a 3D surface. And on the contours of the face they are not parallel anymore. They are getting curved. Based on this it is possible to compute the depth of an accuracy of 0.2 millimeters or centimeters. The right one is you project a little dot to the face. Depending on how the dot size changes they can compute the third dimension. There are several algorithms on how to do actual face recognition. The left one is graph matching. You are looking for facial features which are very well recognizable. And when you turn your head it is still possible to recognize people. On the right side is the so called Eigenface. There is a database of about 100 basic faces. And it works kind of like a phantom image that they use to search for people that you don't know. It is a composition of many basic faces which is computed into one final face. Usually done with machine learning where nobody actually knows what they actually do in detail. How is facial recognition used actually? Or are they used types? You probably know that all mobile phones do it besides the fingerprint recognition. This is the second widely known biometric used in mobile phones. And since Apple introduced the Face ID in 3D you can't only use it to unlock your device. It can also be used to unlock apps for banking apps. Another big application is the electronic password. At the beginning of the 2000s we began to... The third thing is camera surveillance in general. Like identifying people using cameras in the public. There is also a quite new thing. Instead of printing out the boarding pass you can already use face recognition to enter a plane or to casinos or etc. If you don't want to let people that are addicted to gaming into casinos and so on facial recognition is being used to verify these records. So what are the advantages or disadvantages? Obviously a large advantage is that you don't normally need a lot of expensive additional hardware. So all mobile phones or computers these days have cameras activated and for just regular facial recognition in 2D in the visible spectrum that's enough. It's contactless and it's usable over large distances. So the question is if that's a pro or a con. Always have to take the intention of the entity doing the facial recognition into account and a clear disadvantage or counterpoint is the possibility for surveillance and especially with the biometric approach that you don't actually have to consent to it. So as opposed to a fingerprint you actually have to put your finger and put it on a sensor facial recognition can be done without you even being aware of it. As soon as your face is visible. So that's basically everywhere. I already spoke about disadvantages depending on the principle used for recording the picture. It's very much dependent on the environmental lighting. So if the initial photo was taken with great lighting from the front it doesn't mean that it still works when for example the light is coming from the side and at least until relatively short the recognition rates were pretty bad when trying to match against large databases but unfortunately that has recently changed. So I brought an example here this is a bit scary. Unfortunately couldn't get audio working but this is a body cam. So like for example police in America in the U.S. carry these that they carry on their bodies and that makes pictures, video stream faces and then by headphones or speakers they get information that somebody is a missing person or wanted for some crime or other. So this is one of the demonstration videos by Wolfcom one of the distributors of this technology. It's not yet real-time capable but they're working on it. This is just an example video but you can just it's safe to assume that in a few months or a year or so it will be publicly used. You don't give any data on recognition rates and that brings us to a critical point especially if police or any kind of criminal investigation authorities use this technology you should expect the recognition accuracy to be close to 100%. Everybody that's looked into biometric before should know that that's not going to happen. It's not realistic so there needs to be a clear way of how to deal with misrecognitions. This guy said they sold more than 1500 cameras into police departments in many countries and if you have a million cameras on the streets and false positive rates just even 1%, which is completely unrealistic and good, there's still a lot of false alerts. I don't know if you can also see the flicker, it's a flicker here I'm just going to quickly say what's on the slide. This is the status of facial recognition in Germany out of the permeated society. The first step was the terrorism act from 2002 also called the Auto Act after the minister of deteriorated time and acted after 9-11. Of course everybody just jumped on the train wagon and asked several laws regarding everything to security. So in Germany there are 21 different laws affecting things like the passport and press identification laws, the foreigners laws and the asylum laws and originally it called for recording a biometric feature such as a finger or the face and in the meantime they just do both and the advantage should be that you can use computer aided identification and make the documents less susceptible to fraud and finally improving the exchange of information between different law enforcement agencies to prevent terrorists entering the country and of course everybody who's worked in civil liberties should know that there are states with that. Passports used to have side pictures or quarter-ed pictures and they switched in set to a full frontal picture, so the security is actually reduced in case of manual check because for manual verification the shape of the ears is actually very important. So in the past the border control agents could check the shape of the ears and now that you have a frontal picture that is not possible anymore. So if the chip in the passport or ID card doesn't work anymore, security safety is actually reduced and the identification quality goes down. Short note on if the chip is working, the document is still valid in Germany even if the chip doesn't work. Do with that information, that's what you will. Another important step in facial recognition in Germany was an experiment on the central train station in Mainz in 2007, sorry, 2002. So shortly after the terrorism act the BSI, the information security agency after the German government started a study. The first one that actually tested facial recognition at scale and the recognition rates were about 40% back then. There is a funny part of the report that we got back then and it was quite amusing. One was there is a high rate of false positives with people that are thicker in glasses which makes sense of course, so the algorithms that try to focus on facial features these kinds of thick glasses are of course very high contrast, so very important features which means there was a false positive for people that wore these kinds of glasses. So 2007 then central station in Mainz, the recognition rate was a little bit better, approximately 60% in headlighting conditions in the evening especially during twilight in sunset. The recognition rates went down to about 20%, so still not really good. Then the latest field test was in Berlin, Südkreuz station, not so long ago about 2018. Also recognition rates were questionable between 31 and 68%, three different systems were tested. They tried all kinds of weird tricks like dealing systems to each other in different configurations and claim recognition rates of 84%, which is still not really useful. It's actually even worse, but we're going to get to that. One hint for the second phase, they changed. For the first phase they took high resolution images, for the second phase they said, oh the recognition rates are too bad, so they actually took the same kinds of photos that these surveillance cameras would normally use. They used that as a reference image and of course that boosted their recognition rates artificially. But of course it was something that you can actually do in reality. So it only matches in the specific station and if one tried to boost it in a different station then the recognition rates would crater out again. The recognition rate is still about, what was it, about 85% which is still about 15% false positives. These are quite a lot of people if you have 10,000 or 100,000 of people. So you can imagine what happens if actually some people who resemble somebody who is being searched for, was being wanted. On the right side you see another story to the picture on the right side. The second picture from above is a person who was participating in this trial and she was assured that not even the federal police might see these images. It was shown like this on television. But unfortunately this person is not just somebody, but somebody who does documentaries who did documentaries about the rights on their own picture. And now he is preparing a trial about the data analysis. Unfortunately we don't have any raw data. So everything you see here is based on diagrams or graphs that can be shown in the final report. On the left side we have three different colors for the three different systems which were used. What you see on first glance usually all systems should more or less deliver about the same results. But this is not the case here. We see that one system has no result at all in one day which is very questionable. On the right side the two arrows that I marked there is that if you usually have a recognition rate of let's say 80% for the blue one, on one day we only had about 20%. So if I were the manufacturer I would really have to think about it how could such a thing happen. But even worse is that they left parts of the results away. So for about three months they recorded data but for the final report they only took data from about four weeks. So effectively four single weeks but not consecutive and they did not even take the entire week. So it can be assumed that the false positive right has been much worse than it can be seen in the report. Some more numbers of the status quo of facial recognition. In Germany there is a central database with 5.8 million pictures of faces and there were about 24 inquiries to the German Federal Criminal Police. The police had used it about 1200 times and identified 219 people in Bavaria. This is already in operation since modern 12 years. The State Criminal Police of Bavaria has supposedly identified 387 people based on this. But we have the problem that the facial recognition is quite an impact on the personal rights. It should be less easy for a state to do this than Bavaria can just do it. To something more recent, the more recent topic is Corona. But yeah also we have some, in this picture we see problems of facial recognition. These days quite a lot of people are wearing masks and the facial recognition doesn't work anymore. It's clear a major part of the face is covered by the mask. So some people had the idea to print masks with their own face or some other face. So it's still possible to unlock their own phone with or similar things. This one to work with the iPhone it only works with the phones which are working with 2D images. There are some companies which tried to equalize it. They lowered the range which is used for facial recognition. So they only use the eye part of the face anymore. But it's clear that recognition rights cannot get better when such an important part of the face is just missing. A scary example of facial recognition and Corona comes from Russia. They have social monitoring, a social monitoring app in Russia. But they are more advanced than in Germany. It stores biometrics supposedly only in the app. But there were cases where people left their phones at home and were recognized by public surveillance cameras. So there apparently is somewhere a database of people who have COVID-19. And surveillance cameras come in Moscow. They check images from this database. If there is a match you will get delivered to quarantine by force. Which I thought is quite scary so I won't advance into this topic anymore. Another example from China. Facial recognition how it should not work. The story behind it is that in China there is some kind of social scoring. And there is done quite a lot with using facial recognition to do automatic scores etc. And in this case we have usually people cross for instance red light. Then the system does automatic facial recognition and shows it on a big screen like for public shaming. The problem here is that a Chinese businesswoman had a big advertisement on the side of a bus. Of course this bus always crossed when pedestrians had a red light. So the social scoring always thought that this businesswoman always crossed the red light on the street. Which is not good. Let's come to a shitty topic which I tried to do. I did this slide in the last few days but it wasn't that motivated because it's not a nice topic to talk about. What did this company do? This company collected images from several image sources like Facebook, YouTube, Instagram. And they offered to make matches against their database. And in the beginning of 2020 database was leaked with at least 2200 customers. And we saw that quite a lot of countries also the Interpol or immigration offices etc. Were within the customer database of this company Clearview. So there are also customers there where you might think well they don't actually have the right to spy behind me. But yes they actually did it. The CEO of Clearview said that unfortunately data breaches are a part of life in the 21st century. And I find that this is not the right attitude for a company working with such sensitive data such as faces of people. So there was one article I couldn't find anymore that connected the founder of the company to the alt-right movement. And one of the ideas why they founded that company is that they wanted to use facial recognition to mass deport so-called illegalized immigrants in the US. And yeah that's why I just couldn't anymore. But there's also a more positive example or rather an example how you can use this for good. And that is a project that the Center for Political Beauty used in December 2019. There was this one case of a murder in Chemnitz and a mass demonstration by a bunch of schists in Chemnitz. And the Center for Political Beauty then created a website where you can search for yourself or rather where you can search for your friends, your acquaintances to see if they were spot on this protest. It worked pretty well. I had a chance to look at the software running that and was surprised how it worked. So our image is by Chancellor Merkel on a poster that the Nazis were carrying and I just put an image of Merkel into their anniversaries. And at first I thought oh shit this is the point where facial recognition has reached a point where it's actually useful with data at scale. Just to get some more information what they did. They took photos from this protest and video where all these Nazis are passing by an apartment where somebody had a camera running. And then they checked to see how many people they can identify from famous Nazis. Here's one example they found several times with really good recognition rates. And this is where now I said trying to do facial recognition on large groups with bad lighting conditions doesn't work. But now you actually have to say it is different now it's changed now. But I also want to say some more positive things more encouraging things because I'm also looking into how we can work around these kinds of systems. Most of the systems are still pretty easy to get around. So including modern smartphones and facial recognition. You can just use a photo, take a photo with phone and put this in front of the camera and so on. That still works. Of course the manufacturers also know that. And they try to put life detection for example by looking for motion. So motion detection to verify the subject is actually alive so then it's not enough to use a photo what you can play video sequence for example. If you don't want to do that there's another funny trick it's a software that does facial recognition. And that uses blinking for life detection so this is next colleague of mine first to show how it works. You can see the corner the blue eye icon. That said now please blink and then he did and then that confirmed he unlocked the computer. So we wanted to see if we can just trick it. So we printed his photo. This is a print out of his face. And he saw the icons as please blink and instead of blinking we just took a pencil just waved it in front of the photo and it unlocked. So the background is that the life detection works by looking at are there black pixels for the pupil. And are they going away for a short time while you're blinking. And this pen is close enough to skin clearing. So the algorithm thinks that the photo the person close their eyes. There's another method. For example Mastercard uses a similar method that goes beyond just blinking. So they're using gestures and facial gestures to identify people. For example looking sad or smiling. And that's a funny idea here is a video from a U.S. university. So they film the face of a person and then they put it as an overlay over the original image. And then they can make the target say anything or smile or any kind of facial gesture. For three dimensional things it doesn't work. Of course there's also ways to fake that. 3D printers is one that is very obvious. So you have to somehow get a 3D scan of the face. But there's also software that if you have multiple photos taken from different angles that can calculate a 3D model using photogrammetry. Then you can print that. And then you can create a mask as you can see here on the right side. And for simple systems that works very well. Another funny thing I already mentioned earlier that nowadays most facial recognition software is using machine learning. And the American University has looked into how you can confuse these machine learning algorithms. So in the upper row you see classes that they built that they printed in different colors. And these patterns of colors fool the algorithms so much that instead of recognizing the original person's face. It is recognized the famous people that you can see in the lower row no matter which gender. So if you think if it's so easy to confuse these systems then it's even more questionable if they should be used or not. What are the possibilities to flee the facial recognition? A good example gives the table which explains how to do a passport picture in these days. And the other thing which has a red cross means that you should not do it because it lowers the recognition rate. So if you don't want to get recognized do the things which are crossed on these images. For instance turn your head or cover certain parts of your face have hair before of your face. A study said that when you turn your face more than 15 degrees the recognition rate is significantly decreased. Some guy from Japan had a funny idea too. He added some infrared LEDs in his normal glasses. It's not visible to the human eye but the camera is being blinded such that it cannot do an actual picture. I did some more tests using a Samsung Galaxy S8 and I tried out how much I could do until I'm not recognized anymore. So you see simple things like wearing a hoodie or having some hair in your face already makes the system quite less robust. Or you can wear sunglasses, you can wear masks, you can do tattoos or face makeup, special kind of makeup which use special pattern that you paint on your faces. It's even for a human difficult to recognize such a face so the algorithms are getting into trouble even more. So now the last slide I will after that start to the Q&A. There is a dilemma, we are somewhere between total surveillance but there is still a high false positive rate. I always said that facial recognition is not good enough to make master violence but I have learned something better in the last year. It became quite better. Unfortunately these systems work and it's now time to act, it's now time to get active and we need to have laws which says for example it's not allowed to use facial recognition in public areas. This is already be done for instance in California and here in Germany we are still discussing if this might not be a good idea at train stations also. And yeah that was my last slide. I will try to do the Q&A now. So let's see if we have a connection and if there are any questions. Unfortunately I have quite a bad audio quality now so I cannot really translate what they are saying. I don't understand anything here in the translation booth. So I'm sorry we have technical difficulties here, I don't hear the audio, it's very choppy. So unfortunately I cannot translate the Q&A, maybe it gets better in a while, we still wait. If you have other feedback to our translations you can give it on Twitter or Mastadon under the hashtag C3T or under the hashtag C3Lingo on RocketChat. Our website is C3Lingo.org. I will now try to search for the pad so I might maybe at least translate the questions. As I said, there are definitely some things that you might not be able to do by yourself. I will try to search for them from their face recognition and I will try to translate them. But of course there is no way to translate them. We still have bad audio quality in the translation booth. I guess it won't get any better so I'm sorry there is no translation at the moment because we don't understand the original audio. I'm really sorry for that. Have a look at the pad, read the questions yourself and maybe ask somebody from the original stream who might have understood it. The iris or contact lenses might be one idea to improve facial recognition by using multimodal systems. In addition to facial recognition you could add iris recognition because cameras are already prepared for that. At least if you look at modern smartphones the front facing cameras, selfie cameras have a resolution that's high enough, iris recognition. And you don't actually need a very good resolution for iris recognition. So I think we will see systems that combine that face recognition with iris recognition in the not so far future. Okay then there is one question. In Asia the face is used in some part as a public transport ticket. Is there something similar planned for Germany? No, so the Mainz station was a field test by BSI and the Ministry of the Interior in Germany. So it's not at all about actually using it or replacing tickets, transit tickets. It's really just about testing different systems as surveillance systems to find wanted criminals, for example. The test was scaled and in Germany I think there would be legal notifications from privacy point of view. It would be highly problematic to use a facial recognition for that because anything that impacts your basic legal rights less has to be used before any other options. So especially facial recognition, any kind of biometric measurements are very, very personal. It's a very personal feature that is unique to your person, that describes your person. So if you can somehow avoid using that then you should. And I think that is still a consensus for most people. There's still some tests going on. The zoo and handover I think used fingerprints. There was a swimming pool that tested that. And then either very quickly they realized it doesn't work or the people don't use it or the data privacy protection officers actually combined say, well, what's going on? Then there's a question about the false positive rate or generally numbers about accuracy. Yeah, so I realized earlier that I asked that a little bit or rather I didn't actually show that slide. I just need to turn on the slide scene. Okay, so false positives, false negatives. Let's look at this. Okay, let's use this. So before I only talked about the 80% recognition rate, and of course you can look at that isolation, but the recognition rate is always tied to the false positives rate. I'm going to explain it on this diagram, a false positive rate of 0.1%. So every thousand attempt of an unauthorized person is actually accepted as an authorized person. And on the other hand you have the recognition rate, the actual recognition. And here this case, for example, take this logo down with line. And in this configuration, this threshold, if you want 0.1% wrong positives, then you have a recognition rate of 97%. So three out of every hundred people are rejected by the system. Okay, final question. How likely is it that Google or Apple give out facial recognition to have improved recognition rates? Improve recognition rates. Well, so normally the companies use the data themselves, at least Google. The iPhone, of course, they both have their own software, so they have the data. And of course the aggregate data is used to improve these algorithms again. But passing them on to anyone else, they probably wouldn't, because it's also there, they are probably very invaluable treasure trove of data. Of course you can't just do it without getting the consent of the people. So I would assume that they wouldn't do it. But with photos of faces, everybody pretty much does it voluntarily anyway. So if you just search for your name or search for my face, you will find pictures of yourself from events that you visited. And on our Facebook or Instagram there should be a photo somewhere that also contains your name. Or for example, the name of your friend is tagged as label and you can go to Twitter and so on. So you can assume that your face is already there somewhere on internet with your name. If not there, then it's at least with the official authorities. I've got yourself an ID card or passport. The German federal criminal agency has access to this data. The secret services have access anyway. So you might as well assume that at least the authorities, a lot of authorities have access to this data. Thank you and I would say thank you for this. Thanks to you. It's a cool format. It's a bit weird to speak without an audience. I hope it still worked well and I wish you an enjoyable event.