 Hello, everyone, and welcome to our lunch talk on AI accountability, a comparison of methods. I'm here with my friend and colleague, Professor Finale Doshi Vales. And we're going to talk a little bit about a project that we started working on four and a half years ago, I guess. And kind of where that has taken us now in terms of thinking about how to implement methods of holding algorithmic decision making accountable, both from kind of the technical and also from the legal or policy end. It's going to be a fairly informal structure. This is a conversation starting out between the two of us and then hopefully pretty quickly looping in everyone here in the audience. So we thought we'd just start with some quick introductions. My name is Mason courts. I'm a clinical instructor at the Harvard Law School Cyber Law Clinic, where I spend most of my time supervising students who are working on issues of the intersection of law, policy and technology. And I will pass it to finale for introductions. Yeah, so I'm finale Doshi Vales. I'm at the computer science department at Harvard. And my lab mostly focuses on probabilistic models and decision making applied to health care. But as part of that we also focus a lot on accountability in general and explainability and interpretability in particular which is the thing that brought us together for initial collaboration. Yeah, so if you want to start by maybe kind of expanding a little bit on why you got interested in in this issue of explainability and kind of where I know where we as a group were at when we started this. So, I mean, as I mentioned I work on health care applications and here. It was, it was just so abundantly clear as soon as I started working in this space that the data are really messy, we can get all sorts of quantitative measures of how well the system seems to be performing but you know if we've never measured important factors about a patient like how easy is it for them to get to the health system or something like this, then we can actually build tools that don't serve populations very well. And so that was my initial kind of like, oh my we need to understand what these algorithms are doing, especially in certain situations, or it's one tool as part of the toolkit. And then around this time also with GDPR, there was some debate about whether there should be a right to explanation from algorithmic systems, which I thought was a was an intriguing idea where maybe sometimes there should be maybe sometimes there shouldn't be and that was kind of the impetus for this working group on what is the role of explanation when it comes to accountability and AI. Yeah. And I think I came in from a from a related although distinct perspective, you know, I have a little bit of experience in the distant past as a software developer but you know for the last 10 years I've been focusing on the law. And, you know, my interest coming in here was also sparked by the GDPR, partially about the right to explanation but also this discussion that there was this one sentence in there that sparked this huge discussion about AI personhood. And, you know, really trying to disentangle, you know, this idea that automated decision makers could and should be governed the same way that human decision makers were. And I was really interested in exploring different methods of accountability and explain ability was one that really jumped out to me, because it fits so well into many of our existing legal structures where, you know, although there are the US and around the world legal, legal regimes where something is decided purely based on outcome, often the explanation the how is a very important question as much as the sort of the end result. And so it was really interesting to work with a group of computer scientists lawyers policymakers on seeing where the where the similarities and where the differences were in the sort of technical role of explain ability. And then coming from my side, the sort of policy or legal role of explain ability. And that's basically where we got to I feel like you know this, this point from that working group was that, you know, we can demand pretty much similar things of algorithms as we can demand it of people and maybe that's a good starting point because doesn't mean we're asking a lot of algorithms that are also not letting people kind of hide behind the algorithm by saying that hey the algorithm can't do this so you know, you know, all of that legal accountability mechanisms go out the window. Did you want to add to that in terms of the kind of what came out of the report I'm really curious to kind of that's what we were thinking then you know that was roughly where we were then. Yeah, I mean I think yeah that's where we ended up was you're drawing these connections and saying that, you know, explanations are such an important tool in how we govern system is generally like putting aside automated decision making when we think about governing from the level of the individual to, you know, huge multinational corporations were often focused on explanations and, you know, if we have kind of human systems, you know, both at small and large scale like can we extend extend that to automated systems. And I think you I think you know when all you hit the nail on the head when you said it's a really good starting point. I mean I think that is where kind of my starting point was, you know, four years ago but a lot has, you know, I've learned a lot since then that the discussion has evolved a lot and, you know, there is a lot going on you know if we kind of shift from the past to the, you know, to the present like where my thinking is now, you know, is about thinking of different forms of accountability, you know, you could call them different forms of transparency or audit ability as different tools, right, and that's both, you know, technical tools and legal tools and different combinations of them, you know, and I think, you know, from the legal side, you know, we kind of break things down into elements, right, of a cause of action. So we say, well should this have an element of intent should it matter whether you were trying to do something good and maybe you ended up harming someone, or should it only focus on the injury right should it be if you caused harm caused harm your responsibility doesn't matter your intent is intent the same as knowledge right if you were intending something good but you should have known that it was going to cause something bad like what's the level at which we hold someone responsibility so you know pulling these traditional pieces of the legal system of, you know, intent knowledge negligence recklessness, as well as different kinds of harms right one of the things that's been really tricky about piecing these trying to piece together new forms of governance from from kind of like, you know, well it's around Halloween but Frankensteining together right new forms of governance is that sometimes there's not great analogies to me right like so. You know the law has traditionally been very focused on tangible even physical injuries and sometimes you know clearly monetary injuries but you know there's a huge debate right now over like whether just having your date your personal information mishandled is in itself an injury right. And so that's kind of from the legal end the sort of taxonomy or grouping of tools that I've been thinking and I'd love to hear from the, from the engineering side what that looks like. Yeah, and I loved how when we were chatting earlier you were talking about how the legal and you can kind of map these different elements that you talked about to the inputs that go into the system, the system itself, whatever the process that happens, and the, the output of the system like did it actually cause some harm or something like that. And in terms of different technical mechanisms, I think that they kind of fit into those three buckets as well that when it comes to accountability, we first have to check whether the inputs are clean, or appropriate because there's like a transparency is such an important aspect there that the data nutrition projects, you know, through assembly and Berkman is doing some really great work along these lines are just like, you know the data garbage in garbage out right like you kind of need to know what's going into your system, or, you know, maybe it's not garbage but maybe it's just not people who look like me, or something like this and maybe the system's not going to work for a certain population you should just know, right before you don't even need to know the algorithm. And the explainability I feel like really fits into the, the system itself the how right that sometimes you kind of care about the intent of the process of, you know, what did what what what how was it connecting the dots and where was it connecting the dots in a reasonable way. And then there's a variety of mechanisms to check the outcomes. And that's kind of your quantitative traditional performance measures around accuracy and fairness and all of these sorts of things. I think it's also really important to note that these can be applied either or that not either, and they should be applied both prior to the system being deployed out in the world ex ante. And then also monitoring afterwards is really really important because things shift like maybe the population that you train down looks like something but like for example in a healthcare scenario, you know, hospital changes their policies on you know when tests are given their protocols that you know best practices change and all of a sudden your system can't deal with data being collected in a different way or by a different machine or something like that. Yeah, I almost visualize that taxonomy is kind of like a two by three matrix right so you have, you have ex ante and ex post, and then you have the three categories of inputs process and outputs. And I think it's really worthwhile developing tools and techniques for each of those, you know, six boxes, what strikes me is that the legal system is very spread out across those and it's not very consistent right so there are some instances where the legal system is essentially only concerned with, say, like ex post output analysis right so it's like you just measure the harm cause exceeds a certain threshold there's like wait for the harm to happen. Well, the legal system is very focused on waiting for the harm to happen, which is one of the really, you know, difficult issues with, you know, any sort of any sort of regulation but especially in technology regulation is like getting out ahead of problems right it's often not just that you wait for the harm to happen but then you wait for it to happen for the to like the specific person who has the time and interest and resources to bring the case and adjudicate it. You know, and ideally, we have other branches of government that are meant to be more proactive. But those seem to be a little tied up with political things these days. You know, so, you know, I think it's really, it's really helpful to have to look at like the strengths and weaknesses of both the technical side and the policy side of the regulation and maybe see where some, especially the technology can sort up some of the weaknesses in the, in the policy side of what we regulate. Yeah, I do think there's definitely things that technology could do. That's one one other bit that has changed for me in the last several years is that when we originally wrote our, our white paper had our working group having to do with explanation. I think that a lot of explanation mechanisms seem relatively doable at the time, like, you know, like, okay, I could see how some certain things we calculated and as we've seen over the last couple of years. I think that my view has definitely shifted towards it to the extent that you can have models that are fully inspectable by humans. You should because there are so many ways in which explanations explanations that just to provide a broader view for folks who are listening. When I use the term explanation I'm thinking you get a partial view of the system. And this might be quite reasonable because when we ask people, you know, why did you do this or, you know, something like that, we are not asking for every bit of their, you know, brain cells and what what's going on. We are asking for some summary of that's relevant to the question. So it seemed very reasonable to say that hey you don't need to know everything that's going on in a complicated system let's ask for the explanation for the relevant portion. The trickiness comes in is that systems are you know machines are will do exactly what you tell them to do. And so one thing that we've noticed over the past couple of years is that let's suppose that you're building a system that's supposed to provide an explanation over like how features were used to come to an outcome. And you know that certain features should not be used in certain ways like let's say that you're making loan recommendations and you know that you can't be biased with respect to race or something like that or gender. So you tell the system to not be by that you cannot do this right and the way we're going to say you cannot do this is that when we ask for any individual, and you ask for the key features like it better not be the case that the key feature is somebody's gender somebody's race. And then with no additional malicious intent or something in fact this is with good intent you're trying to build the system so that it doesn't have this property. These models can be so flexible that they kind of push the bubble in the wallpaper somewhere else and think, Oh, well if you're measuring, you know, relevant feature in this really specific way, then, okay, I'll build it so it doesn't. So that part is okay. It still could be dependent in another way. And because you never asked the question the system never reveals that part of itself so I think that another piece that has kind of changed for me is that I still believe that the role of explanation is really important but you definitely need to be involved, you know, kind of getting back to the legal and policy side it's not something that you can just say like oh just go and do it. It may not give you what you want. Yeah, and I think for me one of the big things that's evolved, which is also kind of about expanding away from this, the very specific, you know, slice of this that we talked about in our original paper is, you know, I think I still find it very interesting, you know, how do we make automated decision making systems that are amenable to legal to regulation, you know that present enough information to be regulated, but also I mean there's so there are a couple of things that that I've sort of branched out on one is to think about what are the salient differences between a system that is performs performs lawfully versus a system that performs ethically versus a system that performs in a socially beneficial way right because sometimes those aren't the same thing. Yeah. You know, and it's especially true that, you know, I think that ethics tends to be somewhat of an abstraction of our ideas of social good and law is there is a further level of abstraction from our ideas of what's ethical and at two levels of abstraction. I think you might you know as a CS professor you might say like you're getting kind of far away from the original intent and so it's perfectly possible that we will have systems that perform lawfully. That are actually still harmful or vice versa. And I think this is very similar to your that bubble in the wallpaper problem here which is that one of the strengths we talk about with with machine learning and with automated decision making is that, well, they hopefully are designed to find creative ways. So, you know, around creative solutions to new problems. Right, but sometimes they're a little too creative right which is like, Oh, well, you know we'll just, you know we can't use race so and we, you know we can we can't use first order proxy variables like zip code, but, you know, if we can find like someone's favorite football team, you know, maybe we can like find a connection to that. You know so sometimes that created creativity can come around and really bite us even though at that point you're operating in what is illegally a clear zone, you know it might be actually harmful. Yes, yes, yes. I see that there's a couple of questions that are kind of directly related to what we're just saying that have come in. So just to clarify, I that that was just an example of a situation, you know where we, so the question was they are there some situations where it's a good idea to include race to address historical inequities absolutely. This was more of a question of like if the law that says or the regulation say that you shouldn't use this. And you try to create a system that doesn't use it by, you know, enforcing a certain property on the explanation, it might not do what you want so kind of the trying to get at the fact that sometimes there's there's some technical subtleties over here. And we've we've absolutely, you know, explanation is also geared to the to the user rights it might be for a domain expert like we're trying to communicate with clinicians, or it might be for a for an everyday user. So the first question that there I think maybe leads into where we think things might want to go in the future, which is, is it reasonable to expect that should there be explainability requirements everywhere, or for the most high risk systems and maybe I'll expand that question to be just like, where do we want this to go or where do we think that this should go in the near term. And I think, you know, as always there's sort of the in an ideal world where do I want this to go and you know sort of within the operating constraints of reality where do I want this to go. So, I think that there are, you know, regulation can be approached from kind of multiple axes right so one option is to focus on regulation around automated decision making specifically and require certain. You know, certain measures of accountability. And, you know, based on the discussion we're just having, you know, I think that one possibility there is to provide multiple either multiple redundant or at least options of different ways of achieving accountability so, you know, it would be possible to say that all, you know, all systems all automated decision systems must be capable of producing, you know, human understandable explanations that are readable by a reasonable expert in the field or something like that. So you could also imagine a system where you're given your developers are given the option of doing that, or submitting detailed audit data or, you know, or sticking to a limit of, you know, kind of pre cleared inputs right so sort of like, if you use this pre cleared set of inputs you're okay. But if you want to deviate from that then you need to have like satisfy and explainability requirement and or submit to annual audits on the outputs or something like that. I'm not saying that that is necessarily like the best option but like one we could imagine the, I think of that as sort of like a horizontal integration, right. You know, so saying that there are these multiple ways to address that then there's like a sort of vertical integration which is kind of forget a little bit about are we talking about automated decision making or not but just focus on regulating the harms So, so this would be the approach that some people a lot of people are advocating in things like housing allocation right which is we shouldn't really what we should do is focus more on impact and say, you know, if a human is making the decision if an algorithm is making the decision the question is, are we increasing or decreasing housing disparity, right that's what we should be focusing on right we should be focusing on outcomes and less on methods. I imagine that for practical reasons we will probably see intersections of those. I think if you focus too much on methods, you end up by the time the regulation gets passed. It's already outdated. If you focus too much on outcomes, the problem is, then traditionally what has happened there is, you know, you set an outcome based rule but then there's all these edge cases and before you know the edge cases have eaten up the intent of the original rule. So that's kind of what I've foreseen for the future, which is maybe it's less than an answer about what I want what does I see as like potential pass forward. Right and maybe an important piece is just. I think that specific like creating those those detailed checklist is hard because regulation legislation moves very slowly and you want things described at a high enough level that you know it's going to stay relevant and there's not going to have these loopholes or these sorts of things. I think the biggest thing for me is the transparency around like that. I mean there's a set of situations where I think you have to have certain types of requirements and certain types of high stakes and safety critical domains you need reporting requirements around how the system is performing over time. You need to have some you know just like there's a current FDA approval process for devices you need to be able to demonstrate certain things about your, your product before you you put it on to the market. And maybe in other cases, at the very minimum and I'm emphasizing this is the minimum this is not where you know get to, there needs to be a way that the buyer can at least be aware that what way to that that includes not only transparency around the input data which I mentioned earlier, the transparency around you know what sort of accountability checks have been done or are being done for this system, you know, we have checked on these populations and have this level of performance. We have checked to see you know we can expose certain parts of the model to you and you can look at it and there might be multiple ways as you're saying but at least transparency about what has been done can help people at least be again a starting point to make some sort of informed decision. And the reason I mentioned that is that I think that it's so hard to get systems to move. And this was a question that was asked prior, you know, like why is it's hard to implement. But the sort of transparency maybe can become the gold star right like, oh we're a good company because we make all of these things available about our products or something like that and at least maybe you can get some traction that way. You know, certainly, like, be absolutely helpful to keep in mind that like. Fortunately, I think one of the better developments we've seen in sort of the tech sphere in the last 10 years is more people individually getting concerns about concerned about things like privacy security fairness and and and working on those. One thing, one thing that kind of made me think of that I just wanted to mention really quick is, you know, it actually reminds me a little bit of the structure of the Voting Rights Act back when it was whole, you know, and not in its current gutted form where there were multiple layers of accountability right so there were certain regions that were considered kind of high risk essentially because of a tradition, you know, history of voter suppression and discrimination, where if they wanted to make certain preclearance was required. In other areas you could make changes without preclearance but there were still an exposed cause of action to challenge those right and I can imagine a similar tiered structure for regulated decision making where depending on both the history of that particular industry and maybe the foreseeable harm, you know, we might say we might we I think we could reasonably imagine a system of regulation where the degree of preclearance required to change, you know, a music recommendation algorithm is very different than the preclearance for a medical algorithm. Right, you can and those bins I think are absolutely createable right so you could you know from a regulatory standpoint you could create those bins and then there's a question of figuring out what where a system sits and maybe you know after some period of time you realize that maybe the system needs to be in a different bin, and you can move it over as you get a better sense of what harms are possible with that particular application. Well, I see we're about halfway through and we have lots of lots of great questions do you want to pick one and run with it. Sure. So let's see I'm, I'm going to just start by looking at the top. So I think the first one we kind of addressed, which is asking about it. Do we design them for, you know domain experts, I think it's, I mean, explainability is for it for whoever is trying to to use the system. And the next, the next question has to do with what our time tools like lime or shop I'll take this one because it's technical question sufficient for types of legal and policy requirements and there's no. Absolutely not. And the reason why is exactly the sort of thing that I mentioned earlier because like if you have so for those of you who are not familiar lime and shop are specific ways to determine, you know what features are important for a particular decision and they're computed differently and they'll give you different answers actually because they compute different things. So it's already kind of a, you know, depending on the application one of them might be more different than the other. And they have this problem that if you if you train a system that is supposed to be have a certain type of lime output or certain type of shop output. It can do that, but I think what's a what's a simple example that's kind of broadly explainable here's a here's an example. Let's say it's a line with the way line works. Is it kind of looks at the local region around a particular point so it says that here you are, and here your neighbors, and let's try to see like what is a decision boundary look like in just this area right with you and your neighbors. And so let's say the boundary looks like this that everyone on this side likes cats and this everyone looks like dogs that's what the model thinks. And so it says that you know the feature that's most important to you know in terms of separating out this boundary is you know the size of your yard. I'm just making this up. So that seems quite reasonable right like you know oh it seems like if you had a bigger yard maybe I would have thought you'd like a dog, rather than a cat. But remember that we're in these really high dimensional spaces. And so the data lies on this very low dimensional think of it like a sheet of paper and 3d space or even worse you know that the data are very thin structure and it's very high dimensional space. So when you say look around. There's a thin paper, you know data lies here, and then there's around, there's nothingness right and you can spill the nothingness with whatever you want to make the system look like it's being fair in a particular way or not depending on a particular feature so that's slightly technical. But just trying to understand why like these these sorts of tools are not really sufficient. That's not to say that there aren't ways to extract the information that we want to extract, but it's kind of the importance of having an expert perhaps be part of that process. The next question is when I think we should we should both take a stab at answering which is, do we think that there are areas where automated decision systems should just be off limits like no go zones. I'll start out I'll say, I mean as a philosophical matter I could probably make an argument that there's some configuration where automated decision making is permissible in almost any area but as a practical matter right now yes I do think that that is the case. And I think they're, you know, you know there are qualities to look at in terms of where we should just not be experimenting right so one, if you is what's the degree of harm that can be caused up there and if there's an error. Two, what's the degree of consent involved right so if it's an area where people are really knowing fully knowingly opting in to using an automated decision making system and they have a good alternative that concerns me a lot less than if it is being thrust upon them. And I would say, you know, the, the, the third one is whether how amenable or how amenable it is to redress, you know, through current systems we have whether that's, you know, through the law or even reputational versus how much does it allow people to evade traditional responsibility. So I worry about those cases that seem to diffuse responsibility and make it really hard to hold anyone liable. So I'll give an example, but you know so one, one area that is, you know, high risk, and largely not consensual is use of a decision making anywhere in the criminal legal system like that really worries me because that is not something people opting into is something that you know people can you know have their freedom taken away in some in some jurisdictions that can have their life taken away. So that really worries me right it really has the opportunity to magnify existing inequities there as well. And I think that redress is hard to get into like even, even with, even traditionally, like trying to iron out the problems in our legal system or criminal legal system is hard. On the other hand, you might have something like using, you know, if, if an individual wants to, you know, gamble on using an automated decision making to make stock recommendations right that's very risky. I wouldn't be fully consensual I wouldn't want a mutual fund doing that without consulting with everyone, but if you want to use that program individually and try to, you know, shoot your shot so to speak. Then, you know, I have, I'm less concerned about that even though it's high risk. It's consensual. And I do think, you know, that's kind of where we're getting to a gray area, also about like account, you know, holding accountability. I do think that might be a place where purely harm based accountability like if you failed failed to disclose. And you lose money like that's it you're liable it doesn't matter how much money you intended to make. And one of the factors that going to my thinking about where we're where we're where we're where it's safe where it's kind of safe with, you know, you know maybe some extra regulation and where we're just not ready to go yet. Yeah, I think that's a great tixonomy of features and I'll just add from a technical perspective. There are some types of problems that are just very hard to specify, and then end up having all of the properties that you mentioned. So, so not only do they involve kind of perhaps lack of consent, or insufficient opportunity for redress and all of the in our high stakes and all of these things. It's also just poorly defined, right. And machines are not good at thinking about questions that are poorly defined so like in the criminal justice system. You end up with, with, with things that aren't just a number right that you're trying that's the whole point you know these are human beings that you're trying to reason about a lot of different issues at state. And I think that's where like machines are not great. And then machines that are not going to be great in settings that have all of the properties that you mentioned, just a recipe for things going badly. Yeah. Yeah, I think it's another interesting area where the technical challenges and illegal challenges line up. It's also really interesting, is it possible or and or desirable to institute something like as most three laws of robotics for AI, we need an AI police force. Well naturally as a lawyer, my response is why stop at three. No, I do think that I think that, you know, just based on my experience with the legal system is oftentimes it's, you know, it's easy to see to state like a general principle. But then as you dig into it you find more and more edge cases, more and more lack of clarity, I worry about something as simple as three laws of robotics would quickly either get swallowed up by exceptions, or it would take so long to hash out. You know, in whatever adjudicated body you have to figure out what, you know, what are the exact exact extents like, what does it mean to cause harm to a human right is calling a human a bad name like enough of a harm that a robot should self destruct before doing that you There would be all these questions so I do think that, you know, having having high level principles is a good place to start. I think the other big problem there is that, you know, we're living like in an increasingly like almost universally global era, and there are huge amounts of differences about what, what, what are social goals should be what is ethical and what is not ethical. You know, whether, you know, whether even we prioritize individual harms over social benefits so I think that I think that having these kind of goals is is a noble idea, but I think it's going to require a lot more detail fine tuning. So I was going to ask you a follow on question is, you know, on the legal side which is, there's, there's a notion of kind of like, well we have to think like in the US things are very sector specific, you know, how things work in, you know, for for drugs is different than food is different. And then in the EU we have the AI act that's, you know, being developed that isn't the three laws that's you know, many more than three. I'm trying to take a much more global approach and I'm curious what you think it kind of from a legal perspective between these two, two different ways of thinking about this. Yeah, well this kind of goes back to the vertical versus horizontal regulation. I, I think that, ultimately, the fact is that either one is going to require a great deal of kind of hashing out in case by case analysis. I tend to think that it's actually, I would, I would be more, I would be more happy with sort of the layered model right so the idea of like, if you have both regulations that apply to automate decision making and regulations that apply to say, the financial sector the housing sector the medical devices sector. So rather than choose one or the other if you have both you get the both best coverage rates kind of like aligning to polarized. Right, especially when it comes to outcomes that you care about, because sometimes it doesn't matter who the actor is like is it a human decision maker, or automated decision maker that the value is say housing equity or something like you're trying to get to you. And that has to be specified by people there can't be a law that says like you know machines must support housing equity doesn't really make sense. Right. Um, do you want to jump in on the next question. Yeah, I was just taking a look so the next question is my question is about data, which can be biased you to unrepresent. I can be biased you to unrepresentative data. On the other hand apparently objective data may replicate current inequality. And so what could be the solution. I think we just need to be really aware of what those current inequalities are in society, so that we don't replicate them. I think that that there isn't really a way like there are ways to work with bias data from a technical perspective but if you don't know that it's bias, then it's not going to work. I also kind of, this isn't quite in the question, but in terms from a policy perspective, I think that ways to get sufficient trust and build the right sort of data collaboratives are collective so people can, or people companies who are in the appropriate areas can pool data is so important, because oftentimes we're doing worse for populations who need, who could benefit the most, or you know who have been systematically discriminated against. And just kind of, we, we, but we don't have the data from those populations and those populations, you know justifiably are not particularly excited about sharing their data because that you know it has been used against them. So, like, I think there is, there's some naughty issues here in terms of like how do we, how do we get truly representative data and how do we build appropriate governance and trust structures so that people are willing to contribute. Yeah, I'll add on top of that I think it's really important to think very critically about what the data actually is. So, for example, you know going back to the, you know the use of, you know, any sort of data analytics really on say, prime data like we actually do not have data about the Commission of Primes we have data about arrests and we have data about institutions right but we actually can't get data on the Commission of Primes right. If we had 100% perfect reporting on every time a crime was committed, you know, it would be it would be a very different world but the kind of like the idea behind crimes is you don't tell people. You know, so like really like when people talk about criminal data I'm always like, think about what you're what the data actually is right you know and I think that's like the step zero in, in getting away from bias data and that's a case where I just, you know, I with a with a handful very narrow exceptions I think it is possible to unbiased that data right because there's so many of layers of bias between the thing we're trying to measure and the data we have that there's no, there's no backtracking. Yes, like there's certain things that machines can do like if they're if you need to up wait a certain set of events happening to make sure that they don't happen, you know, it might be a rare event but it's really important that this like a medical adverse situation doesn't happen. That's a technical problem you can fix. But if there are too many layers as you're saying between what you're measuring and what you want to do it's just it's not possible. So the next question is what kind of punishments or incentives for following any policy around these accountability measures would be most useful. Who do you see is having actual resources and skills to enforce any policies that might be passed around a regulation. That's a that's a really excellent question because, you know, we can set up like the perfect system of, you know, transparency and rules and regulations but if you can ignore them without any, you know, without any ill effects then, you know, they're basically aspirational. I think right now, you know, my thoughts on this have changed a lot and as usual, the idea is like well it's a kind of a combination of, you know, kind of preemptive regulation, after the fact, redress through a judicatory body like a court and social disclosure. Right now I think I feel kind of most bullish on things like pre like market pre clearance, right that I think that there's a large. I think a lot can be done with requiring software developers product developers to go through certain mandatory disclosures prior to using using an automated decision making system, both because I think that will catch some both intentional and unintentional errors, but also those disclosures will set us up well for those cases where, you know, the system the process doesn't catch something but we then need to go and do some sort of after the fact adjustment. That's a great point right you like if it's out in the world it must have this clearance. It's a very, it's very black and white, but you did the thing that you were supposed to do. Yeah, do you have any other thoughts on that one. I really like that because I, yeah I, I think that otherwise it's, it's kind of hard, but you know what, in terms of figuring out exactly how to enforce these sorts of you're the expert. How do you get people to actually do things. I think the next one is one that you're definitely expert on because it is about technical implementation. Let's go to that of the existing AI ethics guidelines and frameworks are there any that make the most sense to implement technically. I think right now that these are really far apart, right, the specifics of what we implement versus the, like what what is actually, you know, the values that are encoded in the guideline. So I don't think anything is kind of ready for implementation. I do think that there are frameworks that are like I was chatting with some folks in the UK and they're definitely thinking very carefully about like what reporting requirements should be there for like medical systems and stuff like that so I think people are people are getting there. I think there was another question slightly lowered down about given that high risk fields are already regulated would be better to adapt existing laws to address the harms posed by AI rather than create a new omnivist AI law, which we kind of already talked about. But is I think a related question of, you know, many in many of these areas there, there are some really really good notions of like, if you just put a product out on the market. You should be making sure that it's monitored in certain ways and we can adapt that to AI systems. Yeah, I agree with that and just to reiterate like I think that in these high risk fields. So you know if we really do consider these high risk. You know, why not, why not, why not both, right. Yeah. So the next question here is for transparency requirements that Senator on reporting, such as which which of these checks have you done. It seems like it would help to have standards that are generally accepted as legitimate any thoughts on how we start building consensus around what form these standards should take. That's a really, I think a really tough one, you know, coming from the, you know, kind of the law and policy side of things. And then making consensus around what would be a socially beneficial outcome is, is very difficult. So then, then operationalizing that right into specific recording recording standards is very difficult but I do think that, you know, we can look to existing impact assessments. So, you know, there, there we already have some spheres where impact is really important right so in housing, you can bring a disparate impact claim in environmental law, you have to go through many rounds of impact assessment before you can you know implement any sort of large scale project that would have a big environmental impact. So I think that looking at what are the kind of shared standards and to an extent shared norms that we can derive from existing impact based regulatory standards. I mean, even in, you know, drug development right we have some standards about what what kinds of adverse effects you need to disclose, and what is an acceptable level of adverse effects. There's a notion of kind of anomaly detection and or anomaly reporting. So there's kind of the like many sectors have their lists of like, make sure that this chemical is below this level make sure that this adverse event is about you know whatever all of these sorts of things so we have things to borrow from in specific sectors. But there I think another key, and I think there is consensus, you know, at least in some of those sectors are around what should be done. And I think the other key thing is maybe a meta consensus around. Because people report weird things happening, you know, with the system, then there's a there's a process by which that weird thing becomes part of the reporting requirement. You know it's like, Oh, loss of smell maybe that wasn't on our list, but now it's on our list for things that we check for or something like that. Because it is going to be a somewhat messy process so maybe just getting consensus around the meta process is the place to start. Yeah, and I will say for any sort of consensus right I think a an initial question is among consensus among whom right you know so for a drug we might say we want consensus among, you know kind of experts in that particular field right and we maybe don't need national popular unity on you know what what is the level of a significant adverse event, although I think we want some correlation between the official view and the general public view. But I do think there are other areas where consent needs to take in much broader swaths of people right you know. I think, you know, that's another, I think tricky question there is what is the, you know, what is the community that that that sort of consensus needs to occur in. Yeah. Let's see. So I think we kind of addressed the next question. Next, the one after that is for you all read it out. Is it better to use the lens of international human rights law to assess impacts of automated decision making tools, because of its universal and contextual features, for example, the, or that is right to equality and non discrimination versus using ethical principles which are subjective and amorphous, which is fairness. Um, so I think that international human rights law is like I was saying before it is kind of an embodiment and abstraction of what we look at when we think of as universal norms. Um, I think that there are some benefits to using international human rights law which is it's gone through a, it's already gone through a large consensus making process that is not specific to one nation one culture, you know, where you can for example in the US we're almost, we're very far and one side of the extreme in terms of focusing on individual harms right and if we were to set individual harm as the standard for what is a good outcome good or bad outcome with automated decision making we're really exporting that you know that cultural determination, anywhere that these systems travel so I think that one benefit of international human rights law is that it does attempt to be universal. I think one of the downsides of it is international human rights law is notoriously hard to enforce against actors that have significant amounts of power. I mean that's, that's true in any sort of a judicatory system right the more money the more power you have the more likely you are to escape consequences. It seems exacerbated in international rights law to the extent that some people, you know, ask like is international human rights as a legal system or is it just a shorthand for explaining existing power structures. So I think those are kind of to me the trade offs right. I do think that in another big question here is that a lot of the development and deployment and harm caused by automated decision makings, you know I don't want to understate how much harm government misuse can cause but also private use of these systems can be very harmful. And there's a there's a lot of difficulty in bringing private actors under the auspices of international human rights law I think that that is moving in a good direction in the sense that there is more talk of, you know, large multinational companies with sort of almost quasi governmental powers, being at least quasi susceptible to human rights law. But yeah I mean it's it's it's absolutely an area worth exploring. I think that to me what I would concerns me is the the traditional difficulty of enforcement in that area. That makes a lot of sense. So our next question is, do you believe accountability should include lost opportunity. Yes, I think that's a simple answer. But that's a major issue that happens. I mean, small scale like why does my YouTube personalized to like one set of videos and then and then it's so easy to click, click, clicking on, you know, like there there could be a whole world out there and it's a lost opportunity there's really more much more important lost opportunities out there, the personalizing systems of personalized I think this is a big issue. There's a lot of losses of opportunity where people don't even know what the options, what options they might have had available they get trapped in a particular way. I also think about that in the sense of, you know, not just measuring the problems of an automated decision making system in an absolute sense but comparing it to what are we left with we don't implement it. Right. Yeah, of course. You know, and this is, you know, we can, you know, it's really tempting to compare automated, you know, a solution to an ideal world right but we were comparing the solution to the existing world that can change the calculus quite a bit. You know, and I mean I think then that comes down to again like, if you're really focused on impact that's somewhat easier to measure like you can say like okay it shifts the needle, you know, this many points in a direction we like. If you're looking at harder if you're if you're really focused on an intent based system. You know, I find that those kinds of those kinds of decisions are harder to account for do a harm's benefit analysis when you're, when you're looking at, you know, intent versus impact. But again, like it, you know, that doesn't mean we should only be looking at impact, because I think that purely looking at impact a will miss a lot of potentially solvable problems. And also it doesn't line up well with, you know, everyone's idea of how regulation enforcement should work I think a lot of people think that intent is important. I do think that they're like, there is something that we don't like a loophole often that we often think about, especially when I when I think about systems in the health space where we're, we often are like, okay but the system it's just a recommendation right it's not the actor right the doctors making the decision or maybe with stock example. They're making the decision to invest in a certain way, but in a very real sense like people turn their brains off really when you see it when a system gives a recommendation or maybe places one choice above another choice. There are much more subtle versions of lost opportunities. There's gross ones as well. But I really think across the board, there's, there, this is like a real issue of kind of people people stop thinking people stop exploring. We kind of homogenize systems so we lose kind of how we learn new things because we kind of go with it and I think it's, it's not exactly a regulatory question or, but there is a kind of important piece here. But that's kind of important to think about. So the next question is, for the example on stock picking won't transparency become the norm because of market pressure or user demand instead of government regulation. Obviously where fairness is an issue like recidivism or healthcare related applications governments need to mandate it due to high risk. I think that I do think that market pressure will will have will certainly have an impact on how automated decision making systems get used. One of my concerns with relying too much on market pressure is in this kind of almost goes back to the sort of the machine learning side of things is, what are we optimizing for right so oftentimes market pressure maximizes gross value and not things like value distribution. You know, so, for example, if I can somehow take $10 from 10 different people and turn that into $200 the market sees that as a win. And I probably see that as a win, but society might not see that as a win. So, you know, this is a, you know, so this is where I think that depending on the area of regulation what you know depending on sorry depending on the, the, the impact and the, and the potential harms. I think of it's, it's more like a slider, right between complete, you know, complete heavy handed regulation versus, you know, let the market sort it out and for different applications you're probably at different places in there. For example, for example, with stock picking, you know, but I think there are other, you know, questions about whether the market is, we know, in terms of transparency right like so, is it going to provide information that you know, kind of casual investors can use and understand, or is it really going to, you know, be a tool that helps like say large corporations maximize their investments while not helping the person who you know has a few, you know, $1,000 in their investment fund. So I think they're, you know, their equity issues that I wonder about. Yeah, but I mean when market pressures are there, that's fantastic, but many times they don't always do exactly what we want them to do. So I'm noticing that we're about at time. I'm, I just thought that maybe it's useful for us to close out by saying, well, first, I mean Mason it's always fantastic to chat with you. And also for everyone who's in the audience, you know, the two of us are really interested in these issues around AI and accountability we hope this was an interesting discussion and you know if, if you were thinking about these things and are like, oh we'd like other people to brainstorm with or chat with, both of us are really passionate about this space. Yeah, absolutely please reach out if there's, if there's anything we haven't got a chance to discuss that you would like to talk about. Yeah, absolutely. Thanks. And thank you to folks at BKC for setting everything up and hosting you made it very seamless and painless so thank and thank you everyone for attending. Thank you.