 Can you understand me? My name is Taiman Scheb. I work as an artist and as a technology critic and a privacy designer, which ever had I wear kind of depends on where I am. But in the past couple of years, I've been working for the European Union in a project called SHERPA, which was about the future of AI and what issues the European Union should deal with around AI. And as an artist, I was allowed to reflect on that research. And that led, amongst other things, to the creation of this interactive documentary called HowNormalAmI.eu. And this is a website where, in five minutes, your face will be scanned by your own webcam. And then a number of algorithms will look at your face and judge you. And I thought to try it, who here has already done it? Not a lot of people. So I thought, if you trust it, I'm also a privacy designer. So this thing is 100% privacy friendly. It runs 100% locally on your own device. So if you trust me, go ahead for just a minute or two, try it, so you can kind of get a feel of what it is before you continue. Because then the rest of my talk will be a lot easier. So if you kind of just see in the website, and you're like, ah, OK, this is what it does. So I'll give you two minutes for that. Shall we slowly continue? You can try it at home all you want. But at least you got a little bit of the vibe of what it is. So it judges your face, on your beauty, and your age, and all kinds of things. So you'll have seen something like this. And this is what you see when you complete it. So this project uses a number of algorithms, or AI, that I found online that were ready-made, that I just scraped from the internet. So beauty, that was a Chinese algorithm, for example, age, and gender, emotion, et cetera. But there was one that I really wanted to add to this project, that I could not find small enough in my searches. And I was one to analyze your body mass index, or BMI, based on your face. Because I know this is happening, but I couldn't find a small enough algorithm that I could make downloadable in this project. So I decided to make one. So I'm going to take you on my BMI prediction safari, my GitHub safari, as I like to call it, where I go to GitHub, search for a BMI in face, and just download everything that I can find, and then look what I've downloaded. And that's just a fascinating thing to do to stroll GitHub for all the weird stuff that's on there. But before I go into it, I'll have to talk a little bit about what body mass index is. So the body mass index is the ratio between your weight and your height. That's all it is. It's a ratio between those two things. And it's important to understand that BMI is not a universal thing. It's something that you can easily compare between people because there are a lot of factors that influence it. For example, men and women have very different types of BMI that you can't just compare men and women, for example. But also culture is a factor, and your sportiness is a factor. For example, men have just wider faces. That's just a biological fact. But another example is with sports, these two people both have a really high BMI. But one is because he has a lot of body fats, and the other is because he has a lot of muscle mass. But you can just say these people have an equal BMI, so they are equally healthy, for example. So BMI is complicated. At least that's the theory. But if you look at and practice what you find when you go to GitHub, you'll find a lot of projects that aren't taking this as seriously. So you go to GitHub and you download this stuff, and this is some of the things you'll find. This is from a Chinese project. There's a lot of Chinese and Indian projects on there, interestingly. And here already you see that the entire complexity of BMI is just that we're going to put faces in three buckets, and those are just thin, fat, and normal. So all the complexity is, no, we're going to make one universal algorithm and it labels you as thin, fat, or normal. This is another Chinese one, which is slightly more complex, which will put you in buckets of fat, little fat, normal, thin, and very fat, but again, not very nuanced there. So you might be wondering, how do you even measure someone's face? How does that work? Well, most of these projects use these seven ratios in your face, and it's a really quite interesting thing. They start by breaking down your face using a pretty standard AI library to get 68 points in your face. This is pretty much what it starts out with. It reduces your face to this. And then they create ratios like this, like cheekbone width divided by jaw width, et cetera. It's easier if I just show you a little bit. So this, for example, is the width of your face in the middle of your face in relationship to the area of the bottom half of your face. So that's one of these ratios that are like the ratio between those two is one of those factors. Another one, interestingly, is the ratio between the outside lines from your eyes and then one in the middle. So this is something about how big the area above your eyes is, basically. And apparently, if you have a bigger, that must indicate that you have a higher BMI. Another one is this one, which is the bottom half of your face in relationship to the top of your face. But here you'll start to see something interesting, which is, remember this one? There's one thing missing, isn't there? There is no points in that for the top of your face. So what do these projects do to get a point for the top of your face? Well, they just extrapolate it by taking the points that they do have, so they take the points from the corner of your eyes, the middle of your eyebrows, and they draw a line through it. And whatever is the intersect, that's now going to be the top of your face. Of course, you already see the problem with this, which is that if you lower or raise your eyebrows, that has a tremendous effect on where this point is going to be calculated. And since you already know that two of these ratios are dependent on this point, like the one between this one and the eyebrows, raising or lowering your eyebrows has a lot of effects on the score you're going to get with a BMI prediction in this algorithm. So why these seven ratios in the first place? Who came up with this? Well, it was a psychologist called David Kutsche, who was researching whether or not you could predict BMI from your face in the first place. He was researching whether that's possible. And remember this one? This one's interesting because this one is in all the algorithms that I found. They all use it. And this is what David Kutsche has to say about this ratio. He says, the perimeter to area ratio did not correlate significantly with BMI in any of the data sets. So his own research says, don't use this one. It doesn't work. Yet, they still all use it. So my point here is that this stuff is not very scientific at all. The science is very dodgy. So another question you might have is to train an AI. You need data to train it on. You need a lot of things. What do you need? In this case, a lot of photos of people. And of those people, you need their weights and their height. If you have those things, you can weight the height. You can create a BMI score. And you have the face. And then you can create your algorithm. So where do you get that? Well, one of the most popular sources is celebrities. Celebrities have a lot of faces online. And then all you need is their weights and their height. So where do you get that? Well, here it gets very interesting. They get their data sets from IMDB, the faces. And then they get the weights and heights from websites like howtallis.org, which these types of websites are, as far as I can tell, just fan websites where people are guessing the weight and height of these people. Because it's totally unscientific. Because you need for these photos and these weights and heights to be measured at the same time as the photo was taken. Otherwise it's not good data. But with a website like this, you have no idea if the photo and the weight and height are any close to the real thing. So the data is a total mess here. Interesting fact is that Christian Bale is very popular in these research papers on how to check whether or not your algorithm is in any way accurate. Because he played in some movies where he has a very thin face and where he has a very wide face. So if the algorithm kind of does a OK job there, that's a sign that your algorithm is OK. Another popular source of data is arrest records. So in the United States, you can be arrested. And without even being found guilty, your photo will be uploaded online. And these police officers will also measure your weight and your height. And that will also be made available. And I found out, because I was looking through all this code, and I started finding these references to things like inmates in the code. And I'm like, what's going on here? And then I started looking a little bit closer. And I realized that on my hard drive, I had not just code, but I had data sets of these photos. They were apparently also just uploaded to GitHub, which was amazing. I've taken one without a photo here, but this is kind of what this looks like, the police arrests websites, where you get all this data. And there's some very sensitive stuff in here as well, where it will also tell you where someone was arrested and what they were brought to, including like a mental health institution that will just be in that data. It's absolutely insane. So this is kind of what it looks like. I just had this huge amount of photos on my hard drive of arrested people. Another source that's popular is athletes, because again, you have websites full of athletes with their faces as well as their weight and height. So another popular source. But of course, as we saw earlier, people who are athletes have a very not a normal BMI, so it's not a great source to use. But perhaps the most interesting one was this one. Maybe the most shocking is Reddit. So I found a project that scraped Reddit's progress pics community. Progress pics is a community where people share their weight loss progress. So they show it before and after picture, and they always use the same format. So it's easy to scrape, which is exactly what Google Research Health India did. So I got in contact with them. Is this an official Google project? And they said, no, no, no, it's not an official Google project. And someone I knew at Wired, a journalist, also asked them the same question and got a completely different answer. And a few weeks later, the project was gone from GitHub completely. So very interesting. But yeah, they created a scraper that just took progress pics from Reddit to train their algorithm. So I decided to make my own BMI prediction algorithm, because as I said, I couldn't find one that was small enough to use online. And what I basically want to talk to you is about all the design choices I had to make in that. Because when we talk about AI, it can be easy to talk about something objective and neutral. But it totally isn't. It's just a large string of design choices that I'm making. So the first one is, was I going to be nuanced about BMI? Was I going to take gender into account, for example, or culture as I should? But no, I wasn't going to. I wanted to make an AI that was as bad as the one you would find in industry, because I wanted to show how bad it was. But yeah, again, the first choice I made was already to say, fuck it. Then the question became, what data should I use to train my AI? And I had all these photos on my hard drive now from all this downloading from GitHub, including a huge number of Chinese celebrities and a lot of these arrest records. I thought I could work with those two. And yeah, so this is kind of what you get. You get these huge Excel sheets with those photos full of all the names of the photos, as well as the BMI scores. And here, like I said, this is the one from the arrest records. And there, in the top right, you can see someone holding a location was Cherokee Mental Health. So that's, I think, is a pretty serious data violation. Anyway, then the question becomes, in what way do I mix these photos? Like, what would that do to my algorithm? And that was really interesting to try this in different ratios, to see what happens if I made my algorithm with more Chinese celebrities or with more arrest records. And I turned out this had a really big influence. Here you see my AI training itself. It really had, this surprised me how big the influence was. If I put in more Chinese people, the outcomes of my AI were vastly different than if I used more arrest records. So this was really an important one. In the end, I decided to just go for 50-50. I wanted to make, again, a universal AI. So I thought I'd just mix it 50-50. I didn't really have a preference there. Another question that I came to was, will I massage the data? So when you look at all the data points that I had, there were some people who, I'm hesitant to say, were just had strange faces. They were a little bit off in the data set. They were outliers that you will always get when you get a data set like this. And then I came to the same problem that a lot of people who make these AI systems have, which is, will I remove these strange people or not? Do I remove the outliers? Because that has a big effect again on the AI. Because when they were in there, I got this lopsided graph where the majority of people's faces would be measured less precisely because the algorithm was also taking account of these very rare people. So here you see the original data, and this is what happens when I remove the outliers. You see that it becomes a lot more precise for the majority at the cost of not having any data about the majority. So these people are gone. So what's the big picture? How well does this work? Now here you see the prediction error distribution before and after I remove those outliers. So what you see here is I know from all these photos what the score should be. And here you see how many points it was wrong from what it should be. So it should be 14. A large group of people, the score was only two points off from what it should be. So that's kind of good. And then when I removed those outliers to see at the bottom, it became even better. So most people it was close to, but interestingly, when I had those outliers included, the maximum error that you could get is that you would 14 points of BMI score wrong. So there might have been someone with a score of 20 who got 34 somehow. But when I removed those outliers, the people who the system had never seen before who were now also shown to it, the system just had no idea what to do with it. So it got outliers that went all the way up to 20 BMI points difference. So here you see a trade-off that I'm making that by removing those strange faces, those outliers, the effect is that the system works better for the majority of people, but I'm screwing over those people in the majority even more. So those are very real choice I'm making that very much relate to issues in society, about inclusion and stuff like that. So yeah, you see it literally reflected in the AI. In general, you could say that with these algorithms, you have that for a large group of people, the algorithm is good and accurate. Then for a second tier, it's okay. And it goes all the way down until you have about a third to 40% of people for which you could say the algorithm is not accurate at all or even very wrong. And this is kind of the graph that you have with every AI. Every AI will work well for a certain group of people and screw over a small group of other people. So as I said, this was about design choices that I made. Do I go for more gender-specific or do I go make a unisex one? Do I make it culture-specific? Do I make a universal one? Do I make it scientific or not, et cetera? And what you can say is these are basically trade-offs that I'm making. There's no way to make a perfect AI for this. I'm always making trade-offs where if I'm making it better for one group, I will make it worse for the other and vice versa. I cannot make a perfect face recognition AI in this case. That's a very important thing to understand that these systems will never be perfect. They'll always be trade-offs here when you design this. Well, how well does it work? As we saw, it doesn't work very well, right? So as with a lot of these things in this documentary, you'll know that you'll feel like, oh, my God, the system got me totally wrong. Well, yeah, that's the point. These systems are used on you every day and you don't even realize it. But more importantly, they're often wrong, right? But they're right often enough to make it financially interesting for a lot of companies to use these. If your face recognition AI is right 60% of the time, that's already better than tossing a coin, so it's valuable. So why are these algorithms that are basically not working well so popular? So as I said, one thing is a financial aspect, but I think another one that we don't really talk enough about is accountability. One of the reasons you use an AI is to be able to say, I didn't make that judgment. The AI made that judgment. It's a way to offload your responsibility or to evaporate your responsibility by being able to point to something, code, and give it the blame, which I think is a powerful incentive for a lot of organizations to use AI. There's also a lot of things that I didn't put in this system but that are being used, like guessing whether or not you are gay based on your face is a thing. There are some companies who go even further, claiming to predict whether or not you are a terrorist based on your face. And of course, one that I could have literally put in here because I had the algorithm that I didn't was ethnicity. So you can find out or try, they claim to be able to find out someone ethnicity based on your face, but that's again often wrong. So what's my point with all this? Well, I hope to make clear that technology is definitely not neutral. These AIs are not neutral. They are designed. I make a lot of choices when I make it and those choices are trade-offs. It will never be a perfect algorithm. It will always make trade-offs and benefit some groups and harm others. So again, what this means is that technology is political. That's of course what you all understand, but it is, technology is about choices and choices are by definition political because it's something you can discuss together, say, hey, why do you make that choice? Why did you use this data or that data, that methodology or that one? It's something we can discuss and that means that we can hopefully hold it accountable. And I hope that this talk can help us understand how this works, how the sausage is made and some of this stuff and we can all keep this stuff accountable. Thank you very much. That was my talk. I hope you enjoyed it. Well, thank you so much. I learned a lot. It was super interesting. I'm glad you liked it. And if there are any people with questions, there are mics in the path in the middle. So if you want to know anything, I think of course they can approach you after if they come up with something smart. But if there are any questions now. Go for it. Thank you for your interesting talk. You've based your algorithm on face recognition. Would it help if you would extend your BMI prediction algorithm to the entire body, for example? That's a good question, yeah. I only found one project on GitHub that even tried that, interestingly enough. So you would expect so, yeah. But again, I don't think even if you do that, that that will create a perfect BMI prediction algorithm since that's just not possible. It'll still make mistakes. So, but yeah, I think that's interesting why that's not, it seems as if they all just follow the same, follow the leader. You know, like there's one algorithm that tries it and it uses these seven ratios. I think it was a Chinese project that did that. And then just everybody starts jumping on the bandwagon using the same technique and it just becomes the standard somehow. Yeah. Okay, thank you. There is an initiative about banning facial recognition. What do you think about that? I'm a part of it. I created another project. If you like this one, there's another project I made in a part of the reclaim your face campaign that you're talking about, I think. Which is a game where you can try to not be recognized by the AI. And of course, why can't I think of the URL right now? I'll, if you want, I can tell you later what the URL is. But yeah, I think we should ban face recognition technology in public space, absolutely. The consortium, the AI consortium I was part of for the European Union. This was one of our recommendations. Actually, it was one of the recommendations that I had a big hand in. So yeah, thank you. Hello, thanks for the talk. From a more political standpoint, are we to expect the European Union to put enough safeguards in place to kind of balance this out? I hope so. I mean, I'm just an artist working, who works on the European Union research project. So I'm not the European Union. But I mean, yeah, I'm glad I live in the European Union because at least we're trying to do this stuff. Sure enough. Yeah, yeah. Okay, thank you. Yeah, sorry if I missed this. I was sitting in Abacus waiting for the talk to start. So I missed most of it. I'd be interested in the validation of AI software. Do you deal with that? Did you deal with that in your talk? Sorry, because I come from the medical software and we were talking AI, you have to validate everything, make sure that there is no risk in this software. What do you think about that? Are there strategies? How would you do that? Well, I mean, basically my talk was to show one strategy of doing that. By going to GitHub and just looking at what is out there and looking at all these projects and seeing what people are doing. Basically, that gives you a look under the hood of the car and see, oh, so this is how they're doing the BMI stuff mostly? This is what's normal? And then you're just amazed at how much duct tape is involved. And yeah, so this is a way to audit that and in the sense that I'm now telling you, don't trust these BMI prediction algorithms, they're just held together by duct tape, basically. Okay, yeah, thank you. I will definitely re-watch. All right, yes. Yes, I basically have two questions. First one, have you planned any further extensions of this AI? I think you can remember that. And the second question is, in what ways do you promote new upcoming projects to use this AI or wouldn't you promote it? Oh, I would promote it. Oh, by the way, I just remembered a name. If you want more of this, you can go to riu.eu, which is the game where you can try to hide your face. And your first question was if I would extend the project and that's a good question. I'm not going to extend this, because one of the issues with the project that I had looking back is that a lot of people still took the predictions my system made at face value and the message that you cannot trust these systems that are very wonky didn't come across well often enough. So if I would do it another time, what I would do is something different, which is that I would let a number of different AIs judge your face at the same time. And then you can see that they all create a different prediction. So for example, this beauty algorithm that was in this one is made in China, which means it will give a higher score to people who look Chinese. For example, Johnny Depp is way more popular in China than George Clooney, because he has more Chinese-like features. But the way you get it like a Chinese beauty algorithm and a Western one and an Indian one, then you will be able to see, hey, this really is cultured. I should ask the question, which AI is judging me at the moment? Who made it? Which culture was it trained on? That matters a lot. So if I would do it again, I would do that. I'd give you a judge panel in the kind of a TV show kind of thing. Thank you. Yeah, next. Okay, I wanted to ask what did you do technically? Like, how did you do the classifications? Was it a neural network or just some SVM or... It was a... I used Google's JavaScript tensor stuff. So it was the thing you saw, the animation that was in the browser. So I used very basic, just normal training of... Now, is that as a question or you have... I'm not... So it was fine-tuned on... Fine-tuned on fingerprints that you extracted, I guess. Then just some single layers. Yeah, so my algorithm extracted those 68 phase points as well, then calculated all those phase ratios and then did the normal regression type thing and tried to find... Yeah, it tried to do the same thing that I saw in the examples, basically, and then just would spit out a prediction based on what it saw before, what it learned. Okay, thank you. It's a very simple one. Yeah, I noticed one of the datasets that was used is celebrities. Celebrities quite often have plastic surgery. Did you also come across plastic surgery detection algorithm? No, I didn't. Well, I mean, I didn't look for it either, so... But that would be a good... That would be an interesting GitHub Safari to try, for sure. I mean, there are a lot of interesting things. What I find so interesting about GitHub, for example, is that for Chinese, China has copies of their own platforms for a lot of things. They have their own WhatsApp, they have their own whatsoever. But Chinese programmers still use GitHub a lot. So it's a unique insight into what Chinese programmers think is okay. And so India and China are so highly represented in my data that it's just interesting to find out what they consider acceptable and ethical, which is, as you saw at the very first slides, they're not very precise about this stuff at all. It's really duct tape. And I think that's interesting. One interesting thing about this is just the insight into Chinese practices. Oh, thank you. You're welcome. Okay, thank you. I think we'll be rounding up the Q&A. I don't see any more questions, but if you come up with a question, I think they can approach you. Of course. And ask anything. Thank you for the amazing talk. I think we all thoroughly enjoyed it and another big applause.