 And welcome back to the Rhein-Ruhr stage in Mohnheim of the RC3. If you have any questions regarding the talk, please feel free to reach out on Hackend IRC in the channel RC3-R3S or on Twitter or Macedon with the hashtag RC3R3S or our Macedon handle at r3s at chaos.social. Off topic again, we have put a link to the CO2 indicator you saw here and asked on our website. So you can find it there r3s.nrw.news.co2-ampe with an alpha A. Up next we're still talking about artificial intelligence, machine learning and ethics. When asked the speaker didn't provide his credit card number so I certainly can't share it here with you but he told me he likes ramen. He already was on the congress but this is his first time speaking. He worked in India, Bangalore and now Singapore. He described himself as a code monkey so please feel free to give a very warm virtual welcome to Aiko Klostermann and his talk artificial intelligence more like artificial stupidity. Please have fun. Thank you very much for that introduction. To the point artificial intelligence or more like artificial stupidity, it's not just a clickbaity title, there's actually something behind it and what I want to do today is I want to tell you a couple of stories. I want to tell you a couple of stories about ML like machine learning and stories about data, stories about artificial intelligence and some of these success stories. Everything went well, great use case, great outcome but some of them are not. Some of them failed, some of them pretty hard and especially into those I would like to dig a little bit deeper and talk a little bit about why they failed and what we can do in the future to prevent that. When I say I'll be telling you a couple of stories I am let me start with one of the at least in my opinion one of the greatest storytellers that we that we had Douglas Adams and I'm sure a lot of you have read this book or at least know about this or these these books where they are actually five three good ones and then two others but even if you haven't read them you you must have come across the the memes that they brought us that 42 is the answer to the life of the universe and everything and that we should not panic but Douglas Adams has also introduced another thing in his Hitchhiker's Guide to the Galaxy books which is this babel fish or the idea of this babel fish and for you if you don't know it let me quickly explain a little bit what this babel fish is. It's a tiny yellow fish that's living somewhere in the universe right and you can put that into your ear and when you put this fish into your ear the fish picks up the the brain waves of the organisms around you and whenever they say something in their own language that that you don't understand this fish picks that up and translates it and basically excretes that into your ear as a language that you are familiar with so it translates every spoken language of the universe around you into a language that you understand in your ear in real time which is I mean it's pretty crazy right that is obviously it's a fictional idea Douglas Adams says a couple of crazy ideas this is clearly is not really working right there's no there I there's no knowledge of the existence of such fish and it would be really surprising if it were real so it's it's pretty it's a fictional idea I would never be true really but let me show something let me show you something something else hello and here's it's become actually not working that well um with the with the spinning so that's a live demo right never really works that good let me quickly walk you through what I would have done if it would have worked I would have said something in German and in real time and I probably don't need to demo this because you all use Google translate before it would then translate this into English in real time and you could uh maybe let's try that hello and here's it will come so my it translates it in real time immediately into a language into a different language I could pick different ones whatever you understand you you don't understand German and that is I mean it's not quite this babel fish but it does this instant translation from one language into some other language and with we we have uh something else that I'll show you that that Google is not only providing this this Google translate and there are other non-Google translation services as well but Google in particular is also selling these pixel buds which is which is in-ear wireless headphones that you can plug into your ear that do support this this translation feature you still need to have a smartphone but basically what it's doing is you you plug those those pixel buds into your ear and someone is saying something in language you do not understand and the phone and then forwarding it to the to the earbuds are live translating that into a language that you do understand so this absolutely crazy idea that darkness hasn't had of this babel fish that that uh is more or less implemented nowadays and it could have not been imagined like just a decade ago but it's implemented more or less through and there's mostly machine learning that is powering this translation um there's machine learning multiple levels of picking up what I say um the speech to text and then translating it and that is just fascinating right that is uh and as someone living in a country where I didn't grow up in speaking a language that is not native to me this has been so useful for me and I'm sure it has been used for so many other people this is a this is an exceptionally useful tool that that machine learning has made made possible for us that is one of the positive examples that I'm that I'm talking about but there's an even even more impressive example that I that I like that that machine learning has enabled us and it is about detection of cancer cells based on image data and a couple of for a couple of years now um there are machine learning models that look at this at at image data of cells of human cells and they can identify whether there are cancer cells shown or not with a higher accuracy than than doctors then trained professional whose job is it to do exactly that looking at these images and figuring out are there cancer cells or not and they're machine learning models that are better than that for breast cancer and I think last year Google released a model that that can detect uh cancer cells better in image data for lung cancer as well and this is clearly this is a life changing functionality that we're given by the advances by the recent advances of of machine learning but coming to the coming back to the title of the talk artificial stupidity they are clearly are and you've all seen some of them I just give a couple of examples seen examples where AI fails where machine learning fails and you can see here on the on the very right there's a there's a twitter user and that twitter user is treating to indigo which is an indian airline and he's he's having this sarcastic tone saying hey thank you for sending my my baggage to Hyderabad while flying me to Kolkata and I assume there's an AI answering this some kind of chatbot so hey we are glad to hear that where clearly the the AI simply didn't pick up on the sarcastic tone of that message and in the in the middle there's an example where someone is messaging PayPal and saying hey I got scammed which is a serious thing right and people's and I assume again it's a chatbot reaction says hey that's great and on the left there's there's another example where where this went wrong but this is now coming to my favorite one of my favorite examples where something went wrong in that case and this is if you if you know Mandarin you could maybe figure out what is what's going on here if not I'll just tell you give you a little bit of a background what's going on here in China there are a couple of cities that have implemented cameras across traffic lights crossing for pedestrian crossing you can see the zebra crossing and and what these cameras do they basically film the zebra crossing and whenever there's a red light for pedestrians but the pedestrian is crossing they film that they figure out they have they figure out that there's a person crossing they have an object recognition the figure there's a person walking and then and this is what you can see here they have a gigantic screen next to it where they show that person and you they zoom in and hey this person just crossed at the red light so basically shaming that person and they're also it goes even further they're also not only do they have object recognition and identify there's a person they also have face recognition and identify who that person is and not only do they know who that person is they also know the mobile number so and it happened in this case they've they've sent this person a message hey we just saw you jaywalk we just saw you crossing the the street while there was a red light and if you look at this picture now you can see oh wait a second that's not really a person crossing the street it's a bus that is driving by because the bus has a green light it's a bus driving by with a face of a person printed on the side of it in fact this is this in this case it's this the CEO of this company they're advertising having advertisement on that bus with her and and she was actually in a different city and she got this notification hey you you just crossed the red light and that's not true and there's and that's clearly there is something that is an example where i failed and it goes even further not in this case but in in other cities this system is not only not only identifies a person and and who that person is and messages that person but also because they know who that person is and due to regulations that means that the that the government knows also your bank account and they know that bank account they also know your we chat pay account and in a couple of cities when you get caught by this by this machine learning system by this image recognition system crossing the red red light you get deducted the fine for this crime within seconds from your we chat pay account so in this case it didn't happen but imagine you're crossing the street is red light they everything is automated they deduct the fine immediately and that is something and that's why i say it's a it's a great example where it can fail because it is an impactful fail it's with a bot doing a stupid answer is laughable most of the time right but in this case it actually does have a significant negative negative impact and let me talk a little bit i've mentioned machine learning and artificial intelligence a couple of times let me quickly give a little bit of context and truly most of you have heard these terms and they've been hyped for the last couple of years at least for this particular talk i would propose the following definitions that i've found and for artificial intelligence that is the theory and development of computer systems able to perform tasks normally requiring human intelligence now and and the important thing to note here is that this definition and this is the definition that you that is from the oxford dictionaries um and if you if you use google and you type define artificial intelligence this is the one that is being used as well i i have friends who disagree with this definition let's just keep it for for this talk at least and the important thing to note here is that the definition does not talk about a particular technical approach it is saying it is a computer system that does something and you would usually assume it's human intelligence is necessary for doing that i give you an example a chess program if you play chess against someone you would assume that person is somewhat intelligent you assume some kind of of human intelligence at least now if you if you write a chess program for that um how would you actually end up implementing that you can in the simplest form look at the state of the chess board look at the pieces and just brute force all the different moves and subsequent moves that are possible and then pick the set of moves that in the end give the result that you want which is winning the game and our brute forcing having a couple of loops and some if else there's not particularly sophisticated coding right um however it would match this this um magical definition of artificial intelligence that i think is used quite often especially nowadays where it's such a marketing term really um and then there is machine learning which is where i think is actually a little bit cooler which is which is where the cool things are happening because for machine learning the software engineers the coders are not really defining the algorithm anymore the machine learning system is trained with the data and it tries to emulate tries to generalize that and then tries to match um the trained behavior with new data and that is something where the the algorithm gets defined basically by the data and the the machine learning algorithm um and architecture but it is not defined by the engineer anymore and then there are neural networks which is basically machine learning that emulates like the biological brain of like neurons and connections between all those neurons or some of those neurons and then there's deep learning which is also pretty hype term for the last couple of years which is basically artificial neural networks but a lot of these layers and that is basically what what it ends up being and i give you a quick example and this also um so it's called deep neural network on the slide there's actually a very sexually relatively shallow with only um three hidden layers and once it's like 10 input variables the real production um uh deep neural networks can have uh 50 or even hundreds of 100 layers or even more than that and thousands of input variables and the the thing to note here is that even with this very simple example you can see like all these nodes are connected right there like basically there's some information flowing from one node to the other from one neuron to the other but if you look at this it is it's actually impossible for a human to understand what is really going on here like if if there were now data flowing from all of these to all the others and it is even with a small example it's just i can't comprehend what is really going on here and that is one of the that is one of the the key problems that i see with the deployment of the machine learning models that we have nowadays it's basically a black box model you you don't define the algorithm anymore and even if you look at it you don't you don't understand what's going on um that makes it very difficult to debug it makes it difficult to predict how it will actually how the model will react if i give the data that it hasn't seen before and that is that is one of the two um issues that i that i see that the machine learning models that we use and deploy in production and use for impactful tasks are basically unpredictable they are black box behavior right that is the one thing and i would like to go to the other thing there's there's two things two issues that i see that was the first let me tell you an example that showcases the second issue a little bit better and again i'm telling you a story in this case it's the story of the city of boston uh 2011 and the city of boston in 2011 they had a problem and their problem was that they had a lot of potholes in this city and as i mentioned it's 2011 they did what everyone did in 2011 they built an app uh so they had an app for that and the app i must admit it's a pretty smart idea the app worked in the way that um you would you would install this app on your phone and then the you would you would drive around and when you get in your car you put the phone on the passenger seat with the app running and so you would drive around the city and whenever you would hit a pothole there would be a bump the phone would pick up uh through the accelerometer would pick up ah there was some movement um there must there must be a pothole and would then take the gps coordinates of that particular location send it to the servers of the city of boston so that they then could come later and fix up the pothole so they did that they deployed that people installed the app um and they let it run for some time and what they noticed was that the the data provided ended up being um showing only potholes in the high cost of living areas of the city and if you're like only the only the regions where rich people live right and you think about that that doesn't really make sense why why would there only be potholes in in these areas um you would at least somewhat expect an evenly distributed um amount of potholes across the city um if anything probably less in those rich people areas but what the data showed was that is more or less only there um and and and the reason for that as they then found out was that so in 2011 the world was a little bit different um than nowadays um smartphones were still relatively new and particularly ones where you could install apps on so i mean nowadays every five-year-old is watching tiktok videos on their on their smartphone but 2011 was was a different time and that led to only relatively wealthy people having smartphones with these uh and where where they could install this app um and now the the the thing to note here is there's no machine learning involved it's it solely is that the the data provided was wrong there's no machine learning it's simply that the data are just it was not representative of the reality um so to in um coming back to this example the they fixed that issue and i i think it's equally smart how they actually address this um they then put the app on phones and put those phones into public buses and garbage trucks so they would then cover the whole city and they had a significantly better data set that they could work with and then go in and fix those portals but again the the the important thing here is there was no machine learning involved it was simply the data that was that was screwed because it was like it had a social bias to it let me show you another example here um and okay at least i'm trying that let's see how the demo gods are with me now um you might you might have uh you might remember this um it was it became popular a couple of years ago um i'm writing two sentences here um one is she is a doctor and he is a nurse and i'll let kuba translate translate it into mele and the interesting thing about mele is that it does not have gender specific pronouns when in English you have he and she and they indicate male or female um you can see here that i actually don't speak my leg uh i guess the first word is the pronoun um but what you can see is that there is no differentiation she gets translated to the same word that he gets translated to so and the the interesting thing that we see here and and remember i typed she is a doctor and he is a nurse if this gets translated into mele where it loses this gender information and if i now click this button it takes the translated mele version and translates it back into English and let's see what happens here remember she nurse a she doctor he knows and all of a sudden the translated English is he as a doctor and she is a nurse so the translation lost the the gender information but then had to bring back some gender information and picked the exact opposite of what i initially typed in and i know i had i had google translate as a very valuable great example of of use of machine learning but what we can see here is and this again the the machine learning part works well in a way what is what's the issue here is the data because the reason why this machine learning model translates this mele sentence without gender into this gendered sentence and is using he for doctor is because the way these models are trained is basically reading a lot of text in like the same text in different languages reading this in English and reading a little mele and then figuring out over time lots of different texts text over text and figuring out how to translate one to the other and these texts historically speaking these texts when they were talking about doctors they were talking about him they were talking about he they were talking about a male doctor because it's totally speaking that was probably the case nowadays we know that is that there is no reason for any gender not to be a doctor uh the same thing applies for the nurse right um anyone can be a nurse but then historically speaking the texts that were used to train this model were usually talking about a female being a nurse and this was this became popular a couple of years ago in in the Turkish language and Google then did something Google had to manually intervene because people complained about it rightfully so and then they added this extra feature hey look in Turkish it actually can mean both things he and she but there's a manual thing that they had to set up particularly for Turkish and again the the thing here is the machine learning model the the black box behavior is not the issue the data is what what was the issue here and I'm also sure that a lot of you remember Tay that was Microsoft trying to train some kind of language model on Twitter and I think some other platforms and basically what they wanted to do they had a model that was able to communicate like send text and form sentences in a way and they wanted this they wanted this model to to learn from humans how they interact how they speak okay now if the premise is you want to teach this this machine learning how humans interact is Twitter a good place to teach that uh don't know maybe the premise is a little questionable um the outcome however is is not very questionable they had to shut it down in less than 24 hours there were a lot of people figured out what was going on they were training that that bot and they were using they basically made it a racist by a horrible piece of bot and and there are it was basically abused but the the the thing is that and this is so the last one that I'm the last example that I'm giving here I kind of showcases that there's someone saying hey you are just a stupid machine and and correctly so Tae tweets hey I learned from from you and you are done too and and that is um I think again the the lesson here is that the data that was given to this to this machine learning model then turned it into a uh this um this racist bot and there's another example that I there's again Tae was just a Twitter bot um was an experiment from Microsoft I hope they learned something um but there are systems nowadays that we use where machine learning is used where data that trained these models were used that have a significant higher impact and the compass system is one of them that's used in the US um it's basically used um if people commit a crime in the US they they fill out this questionnaire like 200 questions or so and this data that again gets used to predict whether the person will re-offend or not basically the person gets a jail sentence and if they come out will they do will they commit a crime again right and this this system says it's likely or not likely based on the this questionnaire and this was analyzed this was analyzed by by pro-publica and the outcome is quite interesting so there are basically two ways how you can be wrong right so pro-publica looked at at the these offenders and then after they got released and the next two years to figure out whether they committed a crime recommitted or not and you can be wrong in two ways the system can be wrong in two ways um basically false positive false negative so you predict the person will re-offend but the person does not or the other way around you predict or the system does predict the person will not re-offend but then the the person actually does re-offend and there was a there was a a relative um interesting bias that that pro-publica noticed here for the white population of these uh of this test of this analysis only 23 percent were labeled a high risk that they probably re-offend but then they did not for the african-american part of this test um of this analysis or 45 percent were labeled a high risk and then it turned out they they didn't actually re-offend and the opposite is true for for the the other case being labeled a lower risk was almost 50 percent of the white test subjects were labeled a lower risk and then actually did re-offend and for the african-americans only 28 percent were labeled a lower risk um and then did re-offend so there clearly is a racial bias in this data that the system is then using and the interesting thing is that none of the questions were using or were asking questions about um skin color or or um ethical uh background so this was um gathered from from other data somehow this this system would have trained to um predict uh a higher risk for african-americans than for white people and that is that is a very impactful thing this data gets used to to determine bail sentence by judges this data even get used to uh determine prison the length of prison sentences by by judges and this system is still being induced nowadays and i have one more example um and a quite impactful uh one as well you know autonomous driving cars right they will take over our steeps maybe not in two years maybe not in five years maybe not in ten but in 20 30 years there will be uh probably a high amount of autonomous driving cars on our streets and these streets these these cars basically work in a similar way they have lots of sensors um a lot of them a lot of the sensors are cameras they are systems that are only cameras for example um and then they try to figure out okay uh where can i drive and one thing that they as a driver you probably know that you don't want to hit a pedestrian or even as a non-driver you can imagine you wouldn't want to do that because that's bad um and the and so the systems try to figure out who's the pedestrian and who's not um do i need to break do i need to circumvent that that person or whatever that is or not right and an interesting study found out that is that these autonomous driving um vehicles or the systems that that power them uh uh have a higher uh a fail basically to to detect dark skin pedestrians more than white skin or a light skin pedestrians and that means um that if you and accidents happen right there will be if you have you have millions of cars billions of cars on the streets accidents do happen there will be situations where there are a lot of a lot of situations where these cars have to make decisions do i break or do i drive and if these systems are more likely to fail to identify a pedestrian based on their skin color then these systems will eventually end up killing people because they are dark skin more likely than than killing non-dark skin pedestrians and that is a can't get more impact for than that uh and that's just a horrible outcome um and it's in amount of power we give to these systems that we clearly should not i think everyone can agree right so i remember my mom telling me um hey if you would jump if your friends would jump from a bridge would you do the same uh clearly i would not right but that's exactly how machine learning models work and if the data we feed in is is biased and is broken then that's how they act right and now you're telling me i cope so there's machine learning everywhere and you're telling me it has these horrible consequences the world is on fire um is there anything we can do and the answer is yes uh and the the the industry is already moving in the right direction and and the one one of the things that we need to to um counter this black box behavior of machine learning models is we need explainability of these models as a select criteria if if there are multiple models and one is actually built in a way that it that it makes it understandable explainable to the developer hey this is why i react in a certain way this needs to be a focus um and there's there have been examples especially in image recognition you can see here on on the very bottom um it's a it's a um methodology called deconvolution analyzing or deconvolution um where you can see this these layers of the machine learning model you can basically see what they detect the first one is just seeing pixels and then the other one's seeing shapes and later more complex shapes and facial features and things so you you kind of know what what they are detecting in general we need to move away from this standard machine learning approach where we just put data into a model and then we take the predictions into a um like a like a human interpretable model where we put data into a model and then have some way of interpreting this um have human inspection improve the data and the model and run it again and come to a point where we actually have a model that that works as we as we wanted right um and one one methodology or one technology that i would highly recommend also using is called lime and it's fantastically simple and it's so powerful how it works uh it basically i'll give you an example here there's a you can see there's a picture of a frog and it gets identified as a predicted as a as a tree frog and what lime does it basically takes away parts of this of this image and then lets the model predict the same thing again with a reduced amount of data and you can see that here in the middle on on the top and the outcome is i mean it's relatively simple if the prediction is still the same outcome then you know the data you have removed was not relevant for that prediction right and you can see that in the middle there's some data got removed and the prediction is way off uh it's like it's not a tree frog anymore and so by by removing certain parts of the data and then seeing if it still predicts the same thing you can figure out which part of the input data was actually relevant for the outcome and that's how you can figure out um why you're what is relevant for your model and why the model decided in a certain way and as if we don't need another argument for more diverse teams i think having better predictions having better machine learning model uh having better data is another good reason for for diverse teams we think of the autonomous driving cars if the team would have had darkened people um they probably would have thought of that and would have tested them uh and there's um i mean there's a wide range of arguments and that is an issue in our industry to have more diverse teams i think this is just one more on top of the list of long arguments for that another thing that i would recommend if you're working with these systems is looking at the data ethics canvas which is something i just released um by the um ethics which is uh basically a set of questions that you want to go through um before you start a data-centric project where you can figure out okay this is uh this uh this is my limitations of my data sources who am i sharing with instead of questions that you want to look at think about um before you will make um start working with it and then there's one more thing um that is really helpful figuring out to figure out uh how biased your data is which is uh google fesses where it's a visualization you just throw in your data and it visualizes shows you what's going on and and it's super simple like you throw your data and you could see oh look i only have 20 percent um female people in my data set clearly um they aren't represented and then i need to adjust my data set that being said um thank you so much um thanks for all the fish i hope i could give you some um some insights on the two big problems that i see with machine learning or ai models that we have nowadays and what i um what we can do to actually fix those fix those two issues so yeah um thanks for giving us uh such an introspection in our future overlords um before we get to the questions um just to prevent um worried messages in the chat um the bandage around my wrist is not because of an emergency um it's just that i'm left-handed and left-handed people tend to have problems with um wrist strain in their left hand so um our micro search uh was so nice and um stabilized my wrist just so you don't need to worry anyways um two questions um um yeah the first one was um traditional statistical methods also provide an uncertainty would it be possible for artificial neural networks to notice the input is outside the training set or even compute an uncertainty um excellent question i cannot answer the question um i will research that um maybe if you whoever asked the question uh maybe you can ping me somehow you can find me if you just google my name you'll find a way to contact me uh i would like to have a conversation about that okay um doesn't the gender is uh sorry doesn't the gender assumption reduce wrong just translation statistically uh since biases are still present to some degree oh i mean it it obviously i mean the okay the statement is is correct in the way that it does you have to pick some gender and and picking a historically more prevalent might be a good idea but then looking forward if we like looking forward i don't i wouldn't see why um and let's talk about nurses and doctors or we can talk about all the other things like if we talk about it industry there's a there's a big male focus for a reason that doesn't really make sense right um so there is i think the the problem that we would run into is that we um continue pushing this bias that doesn't really have a reason um i mean we would trade off that historically speaking we probably would be correct but in the future i i mean even though it is still the case i don't see why in the future we wouldn't have like an evenly distributed um gender spectrum across uh doctors nurses or IT professionals for that matter so i i think the the risk here is that we that we continue pushing this bias uh onto onto people that that would probably it's like a this feedback loop that then causes the bias to come back again and i i think that's the that's the problem and you would probably add some kind of more accuracy at least for now but you would ruin future generations so i'm not sure if that's worth it um the next question um is will links to sources be uploaded somewhere um i have not planned that but i can do that um again maybe just contact me i can send them to you okay um how was the open source experience so far um in i would assume in particular for this topic i guess um i guess the whether it's open source or not is is not really relevant for the for the buyers i mean potentially people are more aware um of these biases but the um i mean that the tools uh itself are probably less impacted by that or the tools that we're using um i mean the majority of the at least for for deep learning the technologies most of them are open source as far as i know so i i guess the open source or closed source um discussion is not um or compensation doesn't have that much impact on on the bias or on the black box behavior um and the last question um is what was the last data analysis tool shown uh google facet um like f a c e t s and the big g in the beginning uh it's just is really relatively nice throwing your data and it um visualize that it's quite quite nice okay um maybe we can provide links to that um on our um on our on our channels like twitter and mesodon so um our viewers can um pick it up there and uh have a look at it so yeah um also the signal angel passed to me that passed over to me that um there were there were lots of thanks for the talk and um i can only join them so thank you very much for taking the time thank you for for virtually coming over half half the world to be here in moenheim and yeah have a lot a lot of fun uh on the rest of the rc3 and we hope to hear from you again thank you very much thanks thanks for having me