 Tilburg University presents Tijsig Talks, the podcast that keeps you updated on the latest developments in the field of artificial intelligence in just one hour. Welcome to this Tijsig podcast. My name is Pieters Konk. I'm a professor of computer science. I'm here with Eric Posma, who is a professor in artificial intelligence. And Anouche Poury, who is a postdoc researcher in the Tilburg Law School. The topic of today is chat GPT. And the reason I want to discuss chat GPT is the following. About three months ago, chat GPT caught the public eye. It exploded on the internet. People started discussing it, started using it. There was a lot of enthusiasm. And as somebody who knows a little bit about these kind of technologies, I thought, this will last a month and then it will die down. So I didn't see really a lot of value in spending in Tijsig Talk on it. But I was wrong. Actually chat GPT continued to grow. More and more people look at it. We have different companies investing in it. So originally OpenAI developed chat GPT, but Microsoft and Google are developing their own variants of a tool like this. And that's why I thought it might be a good idea to discuss it in particular to explore the limitations and the opportunities of chat GPT and also the dangers and chances that it will give us. So that is what I want to talk with you about. And I think that it's a good idea to start with discussing what chat GPT actually is. And when you work with chat GPT, then it very often says, I am a large language model developed by OpenAI, and I cannot do this or something like that. So, but it doesn't really tell us a lot, a large language model. What is that? And I think I have to look at you, Eric, to talk about that. So can you tell us a little bit about what that actually is and how it works? Yeah, so there's a whole evolution in AI. And it started with the basic large language models, which were trained on huge collections of text using machine learning or deep learning as it is called. And normally what you need to train these systems is, for instance, you have examples and labels. For instance, for image recognition, you have cats, pictures of cats and dogs. And you have to tell the system, this is a cat, this is a dog. And the same with text, you would like to enter text and then label that for instance, this is a good sentence or this is a non grammatical sentence. And it was quite difficult because you had to collect all these labels. So they came up with this idea of self supervised learning, which means that you give, you have a whole sentence or a paragraph and you present part of it to the machine learning, the deep learning algorithm that has to predict the next words or the next words. That was the original idea behind the large language models. And this allowed you to train them on huge collections of text because the labels were already part of the text and the label is the thing you have to predict. So that was the first thing. And these models were already very powerful. But now what this latest development is, is that they put something on top of it that allowed these models to have a conversation, to have a dialogue with the human. And they used humans to train the system or to tune the system with these dialogues using a slightly different learning technique. But in fact, the power of the model is this enormous amount of linguistic information that is connected from the internet and many other sources. And they always say it's a complete internet. I don't know, but it's huge what they use for that. And how you should imagine this model internally is that it's a model with a lot of little knobs that you can turn thousands, but millions or maybe even billions of these knobs that you can turn. And somehow in this model with all these knobs, which control how the input is mapped onto the words that come next, is not transparent. It's kind of a mathematical operation that happens there in these models. And how you do this setting of the knobs is done with a learning algorithm, which takes a lot of time and a lot of energy to find the proper settings. But if you do this for a long time with training and use a lot of computer resources, you get a model that can predict the proper words that come next. And these proper words are based on a huge database of information on text that has been collected from the internet. Okay. I think it still went fairly deep what you were saying. So if I try to summarize that a bit. So basically a large language model is something that can predict the next word based on what came before it. So that's one element of it. So probably you give a prompt to chat. You type something in and tell me please summarize for me to kill a mockingbird. And then it will start doing something because that is that prompt. Will you tell you, okay, the answer should start with this word and then this word should follow. It's probably a little bit more complex than that, but that's basically it. So it's simple ideas. Suppose you store in a huge database all the information on the internet and you ask this question. Can you summarize it? It could search on this in this database if this question is there with the answer because somebody ever asked this on the internet. That would be the kind of naive approach. And that's not what's happening in these models, but it's comparable in the sense that it's abstracting away from these tags. And it's in fact a huge statistical model that alerts to generate the answers that you're, as replied to your question. And those little knobs that you were talking about, they are tuned and they are tuned to do what? To give the proper answer. And what is important to realize is that these models have no sense of the world. Their world consists completely of text. So it's not like humans that read text because if you read a book, you understand the book. Now, this is just a machine that knows if I see these words or these sentences, this is a natural completion of the sentence. Or if I get this question, this is a natural way of answering and not natural in the semantic sense in the understanding sense, but natural in the sense that this is an adequate answer. But that seems to me, and I think that's where we should end with a technology discussion, is if you want to tune these knobs, then you have to tell the model what a good tuning is. So you generate something and then this is the next word that is going to generate, but it doesn't, and actually you have to tell, no, no, that's not the right word. It should be a different word. So then you can tune some knobs and then you get maybe a different word. So how does it know? If that is known? Yeah, for chat GBT there are two things. The first phase is just learning this sentence or paragraph completion. Well, then you know what the answer is because that's in the text, but that's not enough because for chat GBT you have these interactive things, interactive modes. So they use humans, humans that train the system and kind of nudge the systems in giving proper answers to questions so that you can maintain a dialogue. And I think the memory of these systems, they go back in text, I think about 3,000 characters or something like that. So it's also limited in this memory, but it's trained by humans to nudge it in terms of a dialogue. And that's because you cannot learn that from text. Sure. How many humans? I have no idea. It was not this close. At least I couldn't find it. But I guess a lot. Okay. Is there anything you want to add to this? You're making notes. I don't have anything specific on the technical aspect, but I would like to pick up the discussion slightly on the ethical front because that's what I'm interested in in terms of my research. In terms of the humans involved in training of this, I mean, there's a larger conversation to be had in terms of the social license or the social aspects of forms involved in development of disruptive AI. But beginning from a very specific instance, there was this remarkable report, a very moving and horrifying report in time about the exploitative labour practices that were deployed in labelling of data sets which were used for training chart GPT in Kenya where workers were paid for about, if I recall correctly, $2 an hour and their job was to label these horrific images so that the model could learn what would perhaps amount as a hate speech and the model could learn and not spew out venom, so to speak, not to spread misinformation. And that is an extremely troubling aspect. We place a premium on human values. We place a premium on developing products ethically. We place a premium on making AI trustworthy. But usually our analysis is focused in terms of the end user. But all that is happening in the background and the entire development process which as Eric was mentioning is shrouded in the secrecy, the violations which are happening there, I think that should be as much as part of the conversation when we are looking at these models and their impact on society. Sure. And I can imagine especially now other companies trying to develop their own models that they're going to do the same things because they probably know a bit about it. I fully agree with this but you have to realize that these models are no mysteries. Everybody can do this. It's just a matter of resources and both computational resources and humans. So the secrecy is, I'm not even sure if it's deliberate secrecy. It's like Facebook also has these people I think in the Philippines. You have to learn that you cannot allow naked images on Facebook unless it's this iconic image of this girl in Vietnam after a napalm attack which is a cultural thing which is very hard to explain to other cultures. I can see all these obstacles but the only point is that the secrecy does not pertain to the technology itself because that's very open. Everybody knows how you do this but not everybody can. I think we can go back to the ethics a lot in this discussion but one thing that I would like to get a grip on is what chat GPT can and cannot do or tools like that and especially what they can't do because they get ascribed so many possibilities. I leave it to you to come up with an example here because I have plenty of them. If you cannot think of anything then I can't do it. Maybe it's good to start with what it can do because the emphasis is now also in the public domain on what it can do and what the dangers are and the logic and I agree but I think if you look from the perspective of I've been working AI for a long time and that this is now possible that you can automatically generate text that seems to make sense although it doesn't can be a very powerful tool for all kinds of purposes so people that normally write texts they can use this as a kind of first draft of the text. Of course I'm aware of all the limitations there so we have to address them but from a technological perspective I think this could develop into something that is very useful for text writing for coding you know you can generate a kind of draft for the code computer code I mean and I guess in the future you also really saw these movies that can be generated and images so that will be the future the main challenge for and I think that's not a challenge for AI researchers only that's a challenge for all disciplines is how do we deal with all the ethical and practical and cultural problems involved with technology but I see a lot of potential applications and I'm sure that in so I'm not so convinced that this is a kind of hype of course it's hyped but this is just a sign of technology that's moving forward and that will affect our way the way with which we interact with technology so for instance one example I always mention is this idea that you have in these movies of Harriet Potter these talking paintings you know that's something that could also happen in the future that you come home and you have this screen with your favorite character and it interacts with you and can ask you questions and you know that the answers are not always correct and sometimes this this virtual character does not understand our world because it's not based on our world it can be a kind of Google search engine but in a more flexible way that's my picture and that's a positive sign so I see that can benefit many scholars in many jobs I can see how this could contribute I know anything you want to add especially what it can do, what it's good for then we can't do what it can't do I think I'm quite interested in chat GPD and its successor models from the perspective of learning let's begin on the problems that it has raised for all academics for universities and for school teachers and let's see if there is a more positive outcome that we can think of the obvious concern that all of us have at the moment is in terms of plagiarism and I think much has been said about that so I will not address that at the moment but I do see a positive contribution in terms of learning if I could intuitively divide learning the process of learning at three stages in terms of initial introduction and then grappling with conceptual issues of knowledge and finally developing a better understanding I think a large language model like chat GPD particularly has beneficial inputs at the first and the third stage if for example I do not know anything about social psychology and I want just a primer, a general introduction and assuming that the information which is given is reliable then the sustained summary of what social psychology is all about that could be a great way of learning about a new subject similarly for an expert to get a pinpointed answer to an esoteric query that could be of great help but in the medium in the midsection where the epistemic labor is actually involved where we are working our way through the problems and there is this famous thought experiment that philosopher Robert Nozick proposed in terms of experience machine and it was in terms of hedonistic tendencies that if there was hypothetically a pleasure machine that an experience machine that could give us the sense of joy the sense of satisfaction that we wanted after let's say becoming a world-renowned athlete or going through any kind of profound experience actually go through the experience or did not put in through the labor is anything lost in the process and I think similarly an analogy can be made Nozick argues that something profound is lost in the experience because we do not become that person no so similarly in terms of the epistemic labor which is involved the transformation the struggle that we go through in terms of learning before we actually begin to understand a concept yeah but okay so actually what you now say is something that actually then ties back to at the start because you say okay if we yeah I'm in an office and I ask for the social psychology and I get an answer and I can read it nicely say if it's reliable but I think that is the whole point it's not reliable and then the question is it reliable and then the thing is of course that middle part of the process that you start learning then if you if you learn you could assess whether it's reliable exactly and that's what I tell the university people that are concerned about chat GPT I think let students give them a chat GPT essay on a topic and let them score it and indicate where the things are wrong because academics should be able to verify information that's the learning and not that you take over you try to protect it or prevent it from happening because you cannot prevent it from happening and of course the assistants will become better in the future but I would never trust them I also thought okay the thing is if you sent your students home and say write an essay on this topic and you get these essays back and you're not going to read them in detail and you cannot really assess whether they got chat GPT to write the essay for them or themselves however if you say this was generated by chat GPT now indicate everything that is correct and what is not correct then you can grade it again then you can test whether they have a knowledge these are academic skills you have to develop so that's a nice hack instead of us wondering about whether the answer is no no it's not it's and the question come from chat GPT it's not a hack it's a thing that prepares you for the future because these systems are here and they will be there all the time and it's not only the case for chat GPT yet there's a lot of news outlets which the same applies so I would like to briefly address in terms of why perhaps you're right in terms of saying that some of the information which is provided is not reliable and I think there is there are at least two reasons for that one of course we do not know much about the training data set even if we did know the way the output is presently managed it does not attribute sources and once the output is sans the sources it is ethically problematic firstly non attribution of sources we cannot identify who the original author is and secondly it takes away that element of trust that it makes that information less reliable so those are two significant points of improvement for all successive models to come I do see a positive contribution I think at some point in time universities or academia in general would have to come up with grapple with the question of the distinction between learning and assessment and chat GPT and its models are pushing our hand on that but there are some positive in terms of outcomes in terms of learning here yeah okay but if the information is not reliable yeah how can you but I mean I think there is then an issue so because I said what can you do with it and what can't you do with it if you cannot get a guarantee that the information is reliable and is that would that be possible to create a version of chat GPT that is generating reliable information I think not through this route because the whole idea of chat GPT is using lots of data without any knowledge or sense of the world so I always tell students it took you at least 18 years to understand your environment and your culture and that's any biologist or psychologist knows this it's a very complex world and sometimes engineers I don't want to generalize but sometimes engineers don't appreciate this complexity because they think this is a very complicated model and we have Watson and we have this and that but I think you should appreciate this is impossible and if we want to have sources then for instance Google although there might be biases there often if you ask a question you get websites and if it's from a trusted resource you are at least sure that it's something that is scientifically verified or there's another authority who verifies it but these systems don't have that and one of the funny thing was that when chat GPT came up and there was a story that Google was very alarmed by this by the way Google has maybe an equally powerful model itself but the story in the newspapers was that they were alarmed because this could be competition for the search engine and I don't think that's true because it's in no way a search engine it's just an hallucination machine and that could be good for brainstorming or getting out the draft or something but it's not replacing anything that you for what you need human knowledge or at least human verified knowledge and I don't see how you can train these models of course there are attempts to do that train these models to make sure that they always are certified or verified it's a possibility because that's one thing that I was wondering about and I have some idea maybe one of you knows the answer but they indeed say so Google says that the Microsoft says that they want to integrate that technology in their search engine so what do they hope to achieve by doing that because in principle if you use a search engine you get information you see where the information is coming from and you can make a decision whether or not you think that information is reliable so what do we try to do I think the challenge is the dialogue so now we have one shop thing we type in a query and in Google you get an answer and it's not always correct but it's in the proper and people who are skilled in Google can do that but not everybody in the future you have this dialogue maybe a textual dialogue and in the future you have these interactions with maybe avatars or something but I think the dialogue thing is important and what they try to do I guess but I'm not part of Google or whatever is that in each feedback that you get there are links to websites so the integration of this chat GPT like system with the search engine and I think that's already happening if you try Google we'll see that sorry I don't think it's in Google but it's in Binge that the model which is better than chat GPT and they've integrated it and it resulted in some bizarre answers as well I think there was an interesting article in New York Times about the search I think the internal name for this project was given as Sydney if I remember correctly and it was trying to convince the columnist of its personality and its likes and dislikes and whether it has emotional attitudes and whether it likes the user and vice versa and stuff like that so it was a bizarre experience for the columnist and they've taken some steps to mitigate some of these outcomes I think now the queries are limited to five if you're using the chat mode in search it would be interesting to see how in the future instead of and this goes back to the epistemic labor part where you go through the search results and you come up with an answer which in some sense is a mix of your own thinking and learning which has taken place with the help of the search as opposed to this chat this dialogue which is taking place so that is the first major threshold I think which the significant outcome which we can look forward to whether it's a desirable outcome I don't know but it sounds definitely as something that could work that you have a conversation in the conversation you may clear what your questions are and you get the links to the websites which might be an answer to your question which is laid out in some way in the text that you get back to I can imagine something there I think a problem is that you might need to turn more knobs or that you might need to do a different kind of training but I think the issue is here that I notice of chat GPD at least that if you don't give it any hints for what you think the answer is then I will come up with a completely generic text and if you give it hints it starts to confirm what you are saying confirmation bias not automation but that's actually not what you want with the search engine I would say you have a question and you don't know the answer I think I happen to know that Facebook with also Google are actually interested in making this interaction more natural so language understanding hence the large language models implies that you understand the question of the user so now we type in keywords probably in Google and then Bing probably also and others you can make sentences and this could be extended in a more elaborate more precise sentence that's the real value I guess but I think that the rest is problematic because of this I mean for fact finding if you really want to get a real answer to your question these are problematic techniques that's why I see them more as creative things you can use to help your creation process in writing or images or movies or whatever yeah I've seen experiments with that as well so people say okay I want to create a short story I want to create a game and and then generally text is about this topic and you get a text although I still think the text are reasonably bland sometimes there are nice ideas in there but that is in areas where there is no fact it's just all fiction so as soon as you talk about facts and maybe here is then the big problem with something like chat gpt but I would like to hear your opinion that is that the understanding is still in the human so you cannot offload your understanding to chat gpt because it doesn't have it exactly and I think that's one of the problems of the image of AI and many discussions of AI is not grasping this and I think that sometimes pushed by these companies that want to this is intelligent technology and the worst we use we say okay this machine recognizes this picture which is not true it's something else and what we do we recognize a picture and we know everything about the picture but this machine is just mapping visual characteristics on a concept so the nature of what we call understanding is totally different from what's happening in these machines and that's why I emphasize this is called mapping it uses much more information that we can ever use we cannot read everything on the internet but it lacks understanding it has no common sense and that's once you know that and accept that you have different expectations about technology I don't have expectations that it can ever understand our world if you proceed in this way it would require such a system to be embedded in our culture to grow up in our culture and you know from different cultures it's very hard to understand another culture if you didn't grow up in this culture because it's a totally different bias world in terms of the things you learn about how you deal with other people or how the world is organized maybe religion so there's a huge imprint that we have as humans and you cannot compare that to this textual input and even code and images that these machines get from the internet it's a totally different way of learning and acquiring information the question of understanding is an interesting one one way of addressing and somebody who is more specialized in philosophy of language perhaps be a better person to answer this but one way of addressing the issue is in terms of referent when a human being refers to a table the referent is a real object in the world in which the human being is embedded in when a large language model refers to a table it's a probability in terms of a four leg object made of wood is called and that the most probable answer is a word table so in that sense the name escapes me right now but there's a wonderful paper which came out recently in terms of talking about large language models I'm sorry I forget the name of the author and there this point has been made wonderfully in terms of how it would be wrong in our part to transpose our human intuitions in terms of understanding in terms of how we understand conversations and so to that extent I think the question of understanding requires we almost need a different vocabulary to explain the associations the linkages which are happening in large language models these are useful shortcuts to say whether the large language model understands but the human intuition behind that or the intentional stance as Daniel Leonard puts it is not correct one okay so I have lots of topics which we can talk about other things that you want to add on things that you cannot do with it or the limitations or shall we continue with something else? I think in general anything that requires more world knowledge yeah but that is one of the big problems I think because these big companies that is how they try to create interest for their technology and that's at least my impression that they do that but that's on the positives I think there's a lot of potential applications of chat my expectations there will be companies and they're already there for instance you have a Dutch version now you have a Dutch language domain you can automatically generate text for a specific domain you can fine tune these models quite well for instance the text domain or any other thing where you need to generate text and I think a lot of professionals will use this as an extra tool in their toolbox to quickly the copywriters they quickly have to generate the text they ask an outline or maybe an introduction or maybe entire text and then revise it this Dalí model that artists now are you used to be as an artist a good visualizer but now you have to come up with very smart textual prompts to generate I'm wondering something about that also of course I have not seen everything what you can do with it and I've done a little bit of some experience but for instance the generation of text I read by a journalist and she wanted to have an article written and she used chat gpt to get that done and she said but afterwards the editing of that article took me far more time than I would have taken if I would have written it myself I've also seen visual artists use something like stable diffusion mid-journey etc actually I've used it myself to generate some stuff but what I get out of it is it doesn't feel creative it feels rather bland and so if you want to do something quick it's fine but if you say first I make something quick and then I'm going to adapt it then I wonder especially in the visual domain whether you can do that because you start with the wrong thing you start with something that is already black so I was wondering for this visual generation my expectation is that this would just at the beginning there will be better ones that look more sophisticated and the artists are still key it's not that they are replaced by this they will use this in their own way so Photoshop already has these kind of tools and so what artists will do they will use that and they become skilled in manipulating this in a way that gives new art and I'm not an artist but I know they are creative so I see this more as a tool and how they will use it I'm not sure maybe a ride maybe it's more time consuming although there's a lot of copywriters that generate these kind of texts or these kind of images so there's a huge market for it but it is not the highest form of art or writing that that's probably true no but for many things we just need generic art and we just need generic texts so I see and I'm I think in ten years from now we see all kinds of applications that we didn't realize would be possible but I already see a lot of and you know don't know if that will be successful but I could foresee many potential applications of this technology but not in the way that they replace the human factor remains very important so in terms of things that we should not deploy this technology for I think the guiding principle is trust what we trust this technology to do and what we do not trust this technology to do I've co-authored a paper on whether chat GPD can be considered trustworthy with as the Kim Olin who's a professor at Tilburg Law School and with whom I'm working on issues of trustworthy AI and paper is presently under review but we began with reliability in terms of information but the question of trust involves a lot of other things in terms of transparency in terms of human oversight in terms of legal compliance ethical and technological robustness and on all significant issues where it impacts human interests unless an AI system can be considered to be trustworthy we should not deploy it the problem with the hype as you mentioned rightly in the beginning is that ink at breathtaking speed this technology is getting incorporated in ways that we do not understand for example a judge I think pardon my memory recently admitted to using chat GPD for coming out with research and sections of judgment I'm forgetting the name of the country I'm sorry now that to me is of great concern if a lawyer is using chat GPD or any large language model for coming up with first draft of a contract and let's assume that the confidential information the personal information pertaining to the clients have been put into that now that's a significant privacy concern for those clients so in any of these areas or areas where determination of interests have to be made in a judicious manner any kind of government services I do not think we should even remotely consider incorporating large language models there but as we have seen with previous technologies it would happen yes, yes, yes and that is I think the larger ethical concern that we should perhaps address this really is a movement of introspection for all the developers for all the deployers and all people who consider AI as a potential source for good speed with which we are breaking this we are likely to replicate the errors which we have made with other forms of technology for example when social media came first came everybody was really excited with this democratic potential but we have seen how things went at AI now one concern with for example a large language model is in terms of systemic disinformation Rob Dyke and others in their book it's a wonderful book system error they talk about what would happen if the cost of spreading disinformation decreases with creation of synthetic media and with these models that cost is minuscule and if that be the case then we have another problem at our hand so in all if we cannot if we do not consider the system to be trustworthy we should proceed with extreme caution but you also say AI is going to be used for things and that's my own idea about all of AI is actually there is not inherently any danger in all these advanced AI techniques the danger is in people using it for things that it wasn't meant for or people trusting it too much like they take self-driving cars I mean you step into the self-driving car it seems to work you drive around and you keep your hands on the wheel and you do and then the third time think okay this works and you lean back and you fall asleep and then the car drives into a river so that's because people start to rely on it because well it went right they correct the first time it went correct the second time so now it's fine and I think that is where a lot of the danger especially for these kind of tools is especially because they seem to deliver something that sounds like it's reliable I'll just add a caveat to that I think we should first address the question whether the problem at hand is something that should be solved in a technological manner is this something which is a perfect information it's not a game of chess or go where move 37 has existential implications only for all those wonderful players who play that game this has existential implications technologies which are descriptive in nature which has some real life implications we have seen what for example not related to large language models but what deployment of AI system can do in terms of social benefits and discrimination related issues relating to that so the first question is whether technology it all should be involved in drafting solution for the problem at hand second do we have adequate data do we have the right data whether the data sets what kind of biases can we get rid of that bias to what extent can we get rid of that bias is there accountability is there transparency is there human oversight is it possible to contest the outcomes these are all issues which address the trustworthy angle and unless we can address them in a satisfactory manner large language models can only have very limited applications in a wide societal context so maybe I can briefly comment on the bias so anybody working in machine learning knows every model has a bias it is necessary otherwise you cannot generalize it's even true for humans so I think my thing about these biases is that of course you want to avoid biases models it's more transparent what these biases are because you can check it it's more transparent than humans so I always imagine that you would if everybody would be able to test these models and that it would be detected by the community this model is biased in that respect and then as a community has to decide if you want that or not that's now not possible because these models are developed by these tech companies so I think we should move to a situation where we as a community can have a file complaint if we detect something like that and I think that's an advantage with respect to humans because we know that humans discriminate and humans do not behave in an unbiased way so you could also make the case that these models if properly introduced and embedded in a structure that we don't have yet would be advantageous because at least you could check it and verify that they are biased there's no excuse if you test it then you see with this case is comparable to this case and it's only on one factor gender or something and it makes a difference so then you kind of crash test these systems while they are operating I think that's the only solution if you want to use them and I think that's an advantage with respect to humans because nowadays you see that humans when you look at politics there's a lot of discrimination a lot of biases sometimes also unknown biases yes that's a big issue and about this danger of AI I fully agree what you're saying but in fact we're already victim to these things because these little machine learning algorithms behind every news page and every news outlet in social media is effectuating everything you read in the papers that you get these polarizations I think our society is not ready for these systems and that was I think what you referred to that is missed that we were too late but there were warnings that we should be careful and now it's happening and if I talk to young people they like it they grew up with this system but I think we need something in place to make this more regulated or something without obstructing the developments of these techniques because I still believe that they can be very helpful also for attacking problems we face I'm going to slightly take a different viewpoint on this my colleague at law school she's a great thriller she's done wonderful work in the realm of data justice and one theme there is in terms of intersectionality the way we have come to understand our world in terms of that's in different social groups in terms of race gender so on and so forth and this is where the question of bias and explainability becomes at least from my understanding a really huge challenge which we are nowhere close to solving at intersectionality of these multiple focal points in terms of nationality, religion, gender race the way an AI system is computing and coming out with the outcome I'm not sure it is possible to delineate the source of bias or the relevant points of bias and say that this is where we can correct the system and if we cannot do that goes back to the question that I raised in the beginning whether this is a problem which can be or should be solved by introduction of the AI systems in the first place at all so first of all you can detect it finding the source it could have many sources but that's irrelevant it has to be if I am a company and I provide a service and the service is used widely and it's detected or flagged as this is not fair or this is discrimination then it's my responsibility to deal with that by the way there are techniques for that to deal with these things but I think you cannot get an explanation it's doing it before because of that you can repair it but the explanation is hard because all these models have a kind of implicit mapping of all this information on an output in that sense it can be like to humans I always give this example when I was in the old days if you wanted to have a mortgage at the bank and it was rejected a guy in the suit would tell you and give you an explanation why it was rejected and you didn't like it but you went home and you said ok that's a pity and now it's a machine and this machine can at least be checked if it's correct and fair and this guy in the suit might have said that for another reason but you don't know I'm not completely sure about what you now saying because first of all what is bias suppose I say look children have a smaller shoe size than adults that's a bias but it's completely explainable so is this really a bias that is there you can detect it but it's not a wrong thing to have in the system but you also have biases which are because of cultural ideas ok we know lots and lots of examples but the problem with these kind of biases is that you start zooming in on them they become bigger over time because especially with these kind of systems like JetGPT is a bias system we can detect that and it's not weird because it's bias because it is strained on text and a text from the internet and not everybody writes on the internet every topic is on the internet so there's a bias there and then people are going to use JetGPT to generate new text which they publish on the internet so the bias becomes bigger and you should be able to reduce it but how can you detect when a bias is a bias that is acceptable and when is it not acceptable that's why I said at the beginning that this development should not be restricted to AI researchers who study these cases and I could imagine you have a kind of stratification board that is keeping track of that but I'm not saying it's easy but it's necessary because and I think the example that you mention with the shoe size is easy but I think if there is a bias detected that really violates the rules and regulations of the country or a nation and I can see all these obstacles but you shouldn't forget that our society also has these same obstacles as reading the papers, not everybody writes books and still the people that write books have more impact on society so I think this is not only a technological issue it's also a cultural so that's why all the disciplines should be involved here and I think actually Talbot University has a good profile to study this from all sides and to come up with potential solutions. Anything you want to add to that? How long do we have? I was thinking there's probably something that you have a lot to say about. The issue of bias in the AI system I think we should perhaps do a separate podcast from that. There's a lot of forms of bias. Sure. Then round it off. In context of the concerns that we have one way of understanding bias is having an unjustified positive or negative disposition towards a social group or a person belonging to a particular social group. Now these biases are reflected in our daily reality. They get accentuated and multi-applied and they're deployed by AI systems. The example that Eric gave in terms of somebody getting denied a loan application by the man in suit well one way of perhaps making the distinction is that that denial in some sense is on the merit of an individual application that you do not satisfy these criteria. When an AI system is making the competition the model is not just focusing on that individual's data but the computation is taking place in larger socio-economic context and by which of that collaborative filtering then it's not just this person or her outcome which is being determined in light of her life actions but in terms of the larger consequences. Now this is not an AI example but I think this example perhaps makes the point during the course of the pandemic a great level controversy where school children who are about to take their final examinations they could not because of the pandemic and their grades if I recall correctly were a mix of the grade that their teachers gave and also what an algorithm moderated on the basis of the past performance of the school which naturally penalized schools from those areas which were socially and economically disadvantaged and hence they were like us the judge mean not my postcode so when an AI system does that the bias gets multiplied and accentuated in these ways which we do not presently understand so I don't think the and all the quite works in the same way that yes of course there is bias in the real world but the way it gets No I agree because that was not the intention to stretch the analogy that way because it was only that the explainability of a human is not always because that's the kind of gold standard we use explainability of a human but okay you're right you shouldn't stretch this analogy too much it links to something else because it came up now a bit because biases are also culturally determined but these biases currently that are in something like gen GPT are within a particular culture which would mean that if you want to have this technology be used let's say all over the world and for good we can think of good applications then you will have to train it for different culture that's already happening sure and that's all because I also said Microsoft and Google developed their own and there is a lot involved in doing this training you already mentioned all these people that get involved and that's not all practically responsible and there's also things like the ecological impact of creating such models while they still seem to be some kind of play things but I know there is a lot of energy going into these kind of models and it's something that you can say about this so how should we so should we really stimulate the development of all these models for different cultures if you think about them well we're not still in these companies are doing that we do because we're going to use it and that's why we also want to have our model and I think that will happen I'm sure that in China these models are there by Baidu or they are developing it so there are many cultures and that will happen but as I said this is a kind of development phase so the whole AI hype for the last 10 years is built on algorithms that are still in development so every week they become better or improved and the dangerous things that are already applied by companies that really benefit Facebook uses these simpler machine learning algorithms and I guess Google also used these kind of large language models to get more revenue from the searches so I think and that's unprecedented because normally we had something developed and then it was put on the market but now it's already in the market before it's ready and that means that we will have a phase where you have all these variants and cultural variants by the way for cultural scientists it's very interesting to study these models because you might learn a lot but I think again that's something that requires efforts from law makers but also from scientists to study this how you should do this in a proper way I know that the European Union has this ambition to work on a proper implementation of AI and they have this whole leaflet with all kinds of rules and regulations but it's not easy, it's very difficult those are things that they often say it should be explainable it should be transparent these models by their nature are not transparent they might provide some explanations it's like chat GPT they can generate explanations that people like but that doesn't mean that they are not transparent that's a great question Timrit Kapoor and others in their wonderful paper Stochastic where they talk about the financial and environmental cost of training large language models they refer to really interesting aspects where those parts of the world which would face the impact of climate change perhaps will really not benefit from using large language models and it's not even in those languages and I think if memory serves me right there's a form of environmental racism in one aspect I was recently reading this article on Wired where sorry there's just so much information so I'm hedging everything Asked to chat GPT he or it if memory serves me correct then the information there was that to train a large language model like this results in transmission equivalent to a person taking 550 round trips between New York and San Francisco now on the positive side somebody may say that once that training is done well then in the longer run this is better solution for the environment the problem with that is the arms race in terms of who can first monetize large language model who can integrate it with search engine and if this is a cost of training a large language model once then the cost of training it again and again or to run it as part of an integrated search engine perhaps significantly higher so in that sense I think that at least in the shorter and there is a significant environment cost to developing this technology yeah that's an area I think we should be increasing focusing more on interdisciplinary manner I agree with this but on the other hand I think it's a strange situation because there are many things that cost a lot of energy at the same time we see that research and actually a lot of fields of research are having a boost and this boost leads already to all kinds of innovations like better solar panels and it's hard to predict how this will balance out but to focus on one technology of course using so much energy for one such model seems awkward and strange but on the other hand this is technology that goes forward and with this technology I think we need it very hard to solve this actually the most problematic issue of our planet not AI at this moment it's climate change and maybe hopefully at least you already see points that this deep learning might help there by building better models better models better climate models chemistry is held a lot so my hope is that that will help of course there's no guarantee and I think we should be concerned about energy be careful with that but it's a bit of a one-sided story to focus only on these models because it's much stronger okay well that I think that is a very positive view on at least this aspect indeed we shouldn't worry too much about the energy consumption but there are other things that we can worry about things like the aspects of already mentioned education already mentioned the job market although there are positive things there as well but things like how people interact with each other I can imagine that it has impact there how I already see that criminals are using chat GPT to generate spam the spam becomes a lot more natural so I see but that is of course misuse of technology that we are then talking about is there something that we can say about that or that we can do about these kind of things or should we just accept that or I think every technology is open to misuse what is of concern to me and perhaps to many researchers who are working in the field is a lack of conversation this technology these large language models or disruptive AI systems are in some sense inflicted on the society without even initiating a conversation with all stakeholders let's start with academia looking at a sector which was run to the ground where researchers and academics suffered burnt out during the course of the pandemic while trying to do their job to the best of their abilities far exceeding in terms of what was really but still trying to provide and look out for the best interest of their students and they were just beginning to recover from that and now suddenly this new question pops up whether what I am assessing whether that's a legitimate answer or not I do not for one moment believe that somebody who was developing a technology as disruptive as a large language model could not foresee this misuse going forward and you can come up with multiple examples in terms of how creators have been impacted by disruptive AI now just to the other side somebody could say well you cannot have this conversation every time there is a significant innovation but there has to be at least a modicum of a conversation and that's where a regulatory oversight or an ethical oversight I think becomes really interesting because we spoke about jobs it's not a question of what jobs AI systems would replace it's a question of what jobs we want AI systems to replace we do not want our agency to be replaced in a manner which is not of our choosing there are certain jobs which AI systems can do far better than humans and they should and there are certain jobs which AI systems presently are not equipped and perhaps maybe never to address and that's part of that larger conversation that example that you mentioned in terms of criminal use of this technology or systemic disinformation on account of this technology that's a part of a larger conversation that conversation never took place there is this paper on regulatory entrepreneurship which talks about how these big tech corporations exploit this gray area in regulations and then change the regulations almost through lobbying and through their contestation of legal actions and in that sense the technology is not just the training of the model it's also the business model you have taken people's existence capitalized it, surveillance capitalism at its finest and now that's part of a data set and now that's being used for assigning probabilities to tests and we do not know how the privacy and bias concerns have been interested in, the conversation never took place so no but don't such conversations take a long time while the developments in the technology go incredibly fast yes and that's why we usually end up in the mess that we do and then people say I will solve it for us later no I think this is a slow process and we were too late with the upcoming of the internet and now we are again too late because it's so fast but that's actually why we have this conversation here and why we are active as researchers to try to address these things but not by pushing all the time on what's wrong but also seeing what you can do and find a proper balance and there's no easy solution but I always realize that the same technology concerns that are always the bad guys allowed an enormous opening up of our world I mean I feel like grand daddy talks if I tell my students that I had to go to the library and wait for hours for the book to learn something about quantum mechanics and now you press one button and you get all the information I think that's the good side I see these dangers and I think we should address them but we should also be aware of the enormous virtues of having opened up the world and now the question is how do we use this knowledge to somehow deal with that to prevent this mess from happening because I can see that you can read it in the papers every day but I think it's too easy to blame these companies these companies are just the kind of emergent properties of the commercial system and capitalism so in a sense we created ourselves because we gave I have a Google account and of course I have to admit I never read Google and I don't care but you're right they're using information these diffusion models but somebody who found images generated by diffusion models of private medical data I mean these companies just use any data they can use and this has to of course be regulated somehow and that takes lower than I think our regular system is too slow and cannot be fast but that will happen sooner or later and as I said we need multiple disciplines to deal with that okay I liked it do you want to make some kind of closing statement on this yes on the regulatory front there are multiple approaches that one can see at the moment for example the proposed EU AI act seeks to take a more comprehensive approach to AI systems there are a lot of problems with the way it's going about it and I don't think it can effectively regulate large language models there's a wonderful paper by Philip Hacker and others which addresses the challenges of why large language models cannot be effectively regulated under the EU AI act there are other approaches the one that I think is completely wrong is the no regulation approach which US and other countries have adopted UK has an approach where it is sort of trying to address the impact through different legislations through privacy through human rights legislation and so on and so forth the way it is withdrawing from the sphere of human rights with the impact of Brexit I'm not sure whether that will work so I'm not sure whether we have really sorted the regulatory approach but there are multiple approaches which we have working on the problem at the moment so there's hope for the future it's nice to enter the optimistic note well if you would allow me one last cynical take on this I've been thinking about the issue of hope for the past few days and my concern is that anytime we end on a hopeful note people who are perhaps listening to the podcast or somebody who is reading about these topics gets into this false sense of comfort that somebody somewhere is doing something about this and the scientists have been warning about climate change for decades and so many years and that was never paid any heed to in surveillance studies people spoke about privacy concerns for decades but that was never paid heed to so I would say let's end on a moment of introspection but hope for yes in some sense without hope there is never a call for action so to the extent hope is certainly important but perhaps compromise the nature of introspection okay we leave with that thank you very much