 Welcome, welcome, welcome to the first inaugural kickoff session of the MIT Computational Law Reports monthly last Friday series of flash stocks and discussion that we're tentatively calling Idea Flow. And I'm joined by MIT Computational Law Report editor in chief our erstwhile co-host for this event series who is going to kind of introduce himself now and talk a little bit about very exciting updates and new initiatives emerging in the MIT Computational Law Report and then we'll get right into the flash talks and discussion with all of you. So with that, Brian, take it away. Hey everybody, my name is Brian Wilson. I am the editor and chief of the MIT Computational Law Report and I'm really excited today to see everybody's faces after the IAP Computational Law course that just concluded and want to thank everybody who is in the audience who participated there because it was really tremendous and we have some very exciting things that are in the process of being developed in the form of a write up of the course and in the form of some papers and then in a form of everybody's quad chart actually being included as a kind of slide in the final presentations. On top of that though, we're also excited as a publication to announce a little bit more regular publishing cadence that is going to use these community building calls as an input so that we can keep the conversation rolling a little bit more throughout the years and also generate a lot of good impact and get a lot of good speakers in to talk and really highlight the great idea flow that comes out of this group. In addition to that, we are also going to be providing a little bit of a facelift to the website for the Computational Law Report. So there are going to be a number of new features that we're trying out in the form of collections of different ways that content is organized. We're also going to be hosting a number of new columns which are kind of regular series from various contributors throughout the computational law space. And we're very, very excited for those and it's going to be a lot of fun. And I'm so excited that everybody here is going to be able to participate in kind of building that future together. And so with that, I'm going to hand it back over to Diza. Very good. Thank you so much, Brian. And I neglected to mention my name is, as you may have guessed, Diza Greenwood. I'm executive director at the MIT Computational Law Report and a well, a longtime researcher in the area of computational law at MIT. And you can learn more about all of that at law.mit.edu. So now let's do a little screen share. Okay, can you all see this screen? Great. So we're doing a bit of a reprise of some of what we thought was very successful in the recently concluded computational law course. And that is this flash talk segment of content. And we've got two of the three flash talks lined up that you had seen during the class. But what we thought we wanted to do differently and as part of this series, this monthly series is to have the flash talk be presented. So same there, but then actually provide a larger container of time to have discussions that are catalyzed by these ideas and these very brief talks. And so that's where we hope to catalyze the idea flow. And so with that, I'd like, and it's my great pleasure to introduce Megan Ma who's going to give us our flash talk on the legislative recipe, syntax for machine readable legislation. Okay, you're up, take it away. Hi everyone, happy to be here again and happy to see so many familiar faces. Many, if not most of you have heard me speak about this already. So hopefully again today we can have a deeper discussion on it. I contextualize this talk by noting the rather narrow framing, that is I steer away from actually markup languages, standards or the idea of machine readability as an intermediary model, thereby this differentiation between machine readable and machine executable. So without further ado, machine readable legislation has received renewed popularity owe to the rules as code initiative. The fervor around rules as code was accelerated by the recent OECD observatory of public sector innovation report titled cracking the code. This report articulates how machine consumable defined as machine's understanding and actioning rules consistently reduces the need for individual interpretation and translation and helps ensure the implementation better matches the original intent. This methodology enables the government to produce logic expressed as a conceptual model in effect a blueprint of the legislation. So again, what are the attractions and what are its limits? I frequently turn to this example. Lehman E. Allen lamented about ambiguity and legal drafting owe to syntactic uncertainties. In a fascinating study, he deconstructs an American patent statue and notices immediately the complexity with the word unless. He asked whether the inclusion of unless asserts a unidirectional or a bidirectional condition. That is, does the clause mean A, if not X then Y or B, if not X then Y and if X then not Y. There's no nuance to Allen exposes an ambiguity that muddies the legal force of the statue. An interpretation of unless as a bidirectional condition raises the question of what not Y would mean. In this particular case, this could affect whether exceptions are possible in determining patent eligibility. In short for Allen, legislative language must have a clear structure. These ideas are not new. The ancestry dates back to 12th century logicians reflecting on the use of mathematically precise forms of writing. In the mid 1930s, German philosopher Rudolf Karnapp reflected on logical syntax for language. His argument is that logic may be revealed through the syntactic structure of sentences. He suggests that the imperfections of natural language point instead to an artificially constructed symbolic language to enable increased precision. Simply put, it is treating language as a calculus. More recently, Stephen Wolfram made a similar argument. Simplification he states could occur through the formulation of a symbolic discourse language. That is, if the poetry of natural language could be crushed out, one could arrive at legal language that is entirely precise. Machine readability appears then to bridge the desire for precision with the inherent logic and rule-ness of specific aspects of the law. In other words, a potential recipe to resolve the complexity of legalese. However, if a new symbolic language like code effectively enforces a controlled grammar, what are its implications as it moves across the legal ecosystem? In particular, its interactions with various legal sources. Machine readable legislation may therefore be regarded as a product that evolved out of the relationship between syntax, structure, and interpretation. One question is left fundamentally unanswered. What should be the role of machine readable legislation? Is it, A, simply a coded version of the legislation? B, is it a parallel alternative? One that is legally authoritative? Or C, is it a domain model regulation from which third parties derive their own versions akin to an open source code? These three scenarios, and though not exclusive, have their own sets of implications. And only in answering this question would a fruitful assessment of how logic syntax and symbolic language found in machine readable legislation are capable of representing legal knowledge. Thank you. Yay, outstanding. Thank you so much for preparing those remarks. I'd like to remind everybody, and I'm just putting a link in here, that the written remarks are also available for you to pour over and memorize and recite every morning in the mirror at this link that I've just provided. And with that, I think what we'd like to do is jump right into discussion of these ideas, and we'll then go to TMA, who will provide another lightning talk, followed by discussion, and then we'll wrap, okay? So with that, I have to turn to my co-host, Brian, to ask you, how shall we just unmute everybody, do you think, or? The way that I would suggest we go about this is if anybody has a question, you can click the, I believe you can click the raise hand button, and from there we can ask you to unmute so that we can generate the conversation. Very good. And then I might just add one content part of that, which is questions for sure, but also primarily ideas. And so the questions arguably were partly laid out at the very end there with the three options. There may be other options, but if you have ideas on one of those options or pros and cons or none of the above, or other ideas, now is the time to let the ideas flow. And I have a couple of ideas that we can get us started with. Well, actually, first, let me just ask, does everyone understand how to raise your hand? I only learned this recently, but there's a little button that you can hit. But what I was going to start with, and I don't mean to be like rude because the three questions are distinct and they offer different paths, but my basic question, oh, I see Brian has his hand raised. It was just why would it, wouldn't it be more likely that it would be all of the above and that there's different domains of context and use where these would be explored and perhaps others as well, combinations and other ways that maybe people haven't been born yet who are going to make the real jurisprudential insightful breakthroughs about how to combine this technology with law and the fullness of time. But for the next, call it three to five years, not all, what do you think? Yeah, 100%, that's a great question. I'm certainly not saying that it will be one model over another or that it's mutually exclusive. I think that there are sort of nuances behind the implications. So for example, even just between a coded version versus the parallel authoritative, a parallel alternative, it has different implications on the authority behind the machine readable version. So how I see it is in the idea that it's a parallel alternative, that the machine readable and the natural language version are equally authoritative, we might have to turn to ideas of bilingualism or bi-juralism. And so I refer often to Canada as an example, A, because I am Canadian myself, but there has been concern in the past for legislation, say, one written in English and one written in French. So we're conceiving of the machine readable version as another language. So what has happened is that they've seized the courts according to the version that of course benefits them the most, even if there are differences in the types of translation. So this is sort of something that would be a different concern than if it was just a coded version. Because if it was a coded version, then it's just considered an interpretation of the law as opposed to the interpretation by the machine readable version. So that's kind of what I'm trying to express here is that when we think about the various ways in which machine readable legislation could manifest itself, the different, the one, two or three or so there are three different scenarios, they all have different sort of legal conclusions that can be drawn from it. Very good, thank you. And I'm sorry, before we go to Brian, you listening, what was that word you said? Juris prudence, there was some word I'd never heard before and it was like choice among jurisdictions or jurisprudence or something. We'll have to, okay, I'll go back. I think I have a recording. I'm sorry, I'll go back and find the new word. But perhaps others hadn't heard that one before either. I think it was a word where you pluralized different legals. Oh, by journalism. That's it, yeah. Yes, so to explain that, it's in Canada and also as well in the US, there's federalism. So there's the state sort of interpretation versus federal sort of, there's state courts and federal courts. But in Canada as well, we also have two legal systems within the country. So in Quebec specifically, it follows civil law even though the broader sort of Canada is common law. So because of that, it's two sort of parallel systems and sometimes interpretations in a civil law system. So civil law in the same context as European civil law, they have sort of different understandings and interpretations of legal terms. So they're saying that if there are different legal concepts that are in the civil system versus the common system, there might be competition in that regard. So in addition to translation issues quite like straightforwardly between languages, it's also translation of concepts that is at hand. So that's kind of what I'm discussing. Interesting, thank you so much. Brian, you listening, you have your digital hand raised. You're up. Now I have my physical hand raised as well. I don't know why my, do you see this weird circle thing? Snapchat. You've become a circle. Is that a Snapchat thing? You know what, I installed that stupid cat filter and I must have done it. At least I'm not a cat right now. Anyway. You know, anyway, yeah, but we do have good audio, so you can go ahead. So first comment, one is that in the paper and what you just read, you talk about logic syntax, but I think the important thing about logic is that it has a semantics. That's why we care about it. And meaning that, you know, logics have both a, for something to count as a logic, well, first class logics have soundness and completeness proofs. Soundness meaning that, you know, so logics have rules of inference and the soundness proofs mean that everything that you can derive by those inference rules, if the premises are true, the conclusions are true and completeness that everything that should be derived is derivable by the inference rules. And so that's what's important about having a logic. It's not just another syntax, another formalism. We're not just translating, you know, English into, you know, Excel macros or something because those don't really have a semantics. Logics have semantics. That's why we care about them. But the other thing is I was making asking, I'm just gonna ask a question that maybe. Yes, please. It's not the legislation itself that should be translated into logic, but and where people have an opportunity to review this is, you know, the rulemaking process is done by the agencies that implement these things. And there's generally a public comment period and that would be a great time for them to say, okay, here's how we're formalizing this legislation as code. And then the public has a chance to do essentially a code review and say, oh, you know, you didn't quite get this part right or this, there's this exception that you didn't handle here or whatever. And rather than relying on the legislators to do this, which, you know, I doubt that they're capable of, but at the agency and rulemaking level, I think this is much more likely. Yeah, 100%. I think those are fantastic comments. When I speak about logical syntax, I think I refer actually in the very narrow sense that's been purported by Rudolph Karnapp, who had the specific idea that, you know, we need to use symbols in the same way that sort of math has the symbols to represent concepts. So it's almost a removal or a pulverization out of semantics. So I do recognize, of course, that logic itself has this important component of semantics. Here, I turn specifically to kind of this notion that you're just manipulating symbols in no sense. And this is also something that Lehmann Allen tried to do in a later piece where he tries to use symbolic logic for drafting. How I see it is that he's proven actually that you can increase the clarity, but it's more in the sense that it's a metric to measure how clear legislation is as opposed to use for drafting from the start. With your second comment, I certainly agree. I think that there should be openness for people to comment, and especially how, I guess, how lobbyists in many ways go about trying to make changes in legislation. I think some of the interesting things at this point is that rules as code suggests that there should be everybody in the room. So beyond legislators, there should be programmers, and that there's sort of a combination of minds in a way to produce this type of legislation. So sometimes, even, so this is an exercise in parallel drafting, so they're saying that the methodology will enable these comments and that interpretations will already accommodate for other perspectives. But sort of correction and being able to edit the legislation at a later day after the initial draft is also an important consideration. Cool, thanks. Sorry, I was on mute. We're now creating our own little common law of this new series with the following practice and custom. After you have paid the table stakes of raising your hand and posing a question or idea, you've earned the right to not be muted again and to continue to the conversation. Also, I'm not sure how to mute people after I take them up, but you're permitted to mute yourself if you want to, Brian. Okay, and so now next up we have Brendan Maher. You're up, let your ideas flow. Great, can everybody hear me? Megan, wonderful talk. I have two quick points and then I'd like to comment on what you had presented. First to continue on with Brian Ulysses' comments about completeness. This is actually really important. And it harkens back to the constructs of a Turing machine that is finite. And these are very important things to look at in terms of building your models in terms of not just a legal encoding, but one which is complete and comes to a finite state. So those are very important things to keep in mind. Two, I love your idea of, I forgot the exact word there, by jurisdiction presence, I'm not sure what that was. By dualism? By dualism. We're gonna need that when it comes to merging traditional law and space law. So I wanna hear a paper from that about that. But to comment on your main body discussion, what is it important to think about is the finite set of these operators. And you chose a word in your example, if you can refresh my memory, that would be helpful. What was the word you used in your example? Unless? Unless, yes. So in the scope of legal documents, there are going to be roughly more or less a finite number of these words which are how do I say ambiguous, right? Now there will be many more in the space and those will be at the tails of the curves, but generally speaking for most documents in specific areas, there's gonna be a finite set. The question becomes first, how do we identify these finite set of terms for a number of documents and then be looking at the content of the documents themselves and looking at how to take those contents of clauses and essentially from a computational standpoint, bring them into hash table. So like putting them into different buckets. And from that, you can build all sorts of models. And I'm sure you're thinking about this. If not, we could discuss. Yeah, that's actually really fascinating because I actually had a response from a legislative drafter when I talked about this and she had mentioned to me that actually even in natural language and when you're drafting legislation, there's already a finite set of operators or a finite set of vocabulary that they have to use and that they're not allowed to introduce vocabulary or terminology outside of this set. So I think it will be interesting and sort of comparing finite operators almost to see and identifying actually specifically where has ambiguity existed. And a lot of times there's ambiguity that's unintentional versus intentional. Most legislative drafters are of the idea that unintentional ambiguity is what exists because there is already this finite operator that intentional ambiguity isn't really necessarily there. But at the same time, this is a Canadian perspective. In the US, I know that intentional ambiguity is sometimes incorporated just to defer to authority and defer responding at the moment. So I certainly think that your suggestion and kind of looking at these finite operators, that is something that we certainly have to do. And I think that there's going to be going forward almost a matching process. There isn't such thing, I think, as perfect translation, but at least there might be a potential conceptual matching between finite operators computationally and in the natural language when it comes to legislative drafting. Thank you, Dan. I think you're... Sorry, I'm on mute. This is like the plug of our age is you're on mute. Renita, you have your digital hand raised and you now have the floor. Thank you. Thank you, Megan. Your comments are fascinating. I read your remarks briefly and there's so much information to unpack. One question I had was regarding what you mentioned about the two modes of federal and state laws by journalism. That you see in Canada and in the Quebec province. And I'm wondering if that system is working well and if there are specific cases there that show how it is actually advantageous to have two kinds of laws running at the same time. Would it not be the same for certain aspects of machine readable legislation that we have many translations? And perhaps in some niche area, it would be beneficial to have multiple translations running at the same time. For example, on the phone, I know that there is the Bible app and there are so many different translations that people can download and read. And different groups of people, gravitate to different translations. The gist might be the same, but there are some nuances that are manifest better in certain translations of others. And I'm wondering if machine readable legislation might in fact benefit from having multiple interpretations in some cases. Yeah, that's an excellent question. Thank you very much. I think to answer, I will go back to the biduralism question after, but to answer first multiple translations and the benefits of having that. That's in part why I raised the three models or how I see the three scenarios. And the reason is because in the first scenario, if it's just a coded version, it is an interpretation or a translation. So then it would add to sort of the benefit of having multiple translations because it does perhaps increase accessibility in many ways. And that's perhaps why there are multiple translations in, for example, the Bible where you have English, Spanish, French, Chinese, maybe. But I think that having a different translation and having additional translations is different than having an extra authoritative document. And that's sort of where my distinction is and where I talk about the parallel alternative. And that's sort of what leads me into the biduralism discussion. So in Canada, they have had efforts because it was sad to say that before there were just makeshift equivalents and, you know, attempts at making one-to-one translation in a way, but it's almost in many cases. It was, I read it. It was an English version that was drafted. And then somebody who knows French, who is just a translator without legal knowledge had just, you know, converted it. And at times, legal concepts don't have the same conceptual similarities as we think they do. So some things that they've tried to remedy is actually McGill University. There's a new department called Juralinguistics where they actually focus very, very closely on conceptual, integrating conceptual and linguistic translation together, which is why they've sort of merged the name Juralinguistics. And that's sort of an area where they've tried to look at what exactly are parallels in different legal systems and how are we going to improve these translations so that going forward, if there are disputes that people are not necessarily being very strategic about which version that they're bringing to the court, that there are, you know, at some places, a level playing field, I guess, between versions. So I'm not sure if that answers your question fully, but thanks very much. Great. And I'm going to take the co-moderators privilege now to make a comment also about this topic before we move on to TMA. And so just weaving together a little bit, Brian Ulyssany's observation about in the U.S. what we call NPRMs or notice of proposed rulemaking when a regulatory agency is forced to publish a draft of regulations in advance and provide notice and opportunity to comment. It seems to me that, so I'm sorry, weaving that together with what was just said now about multiple different ways to kind of do the matching. It seems to me that there are multiple ways, partly because there really isn't one translation. There's somewhat different domain spaces. Well, profoundly different in some cases. And that means when we come to the area of public law that there are fundamentally political questions about what are we trying to express and how do we express that. And so we may be trying to express and expressing certain ways, certain things in natural language and certain other things in the realm of code as it emerges. And these things must be debated and there will be alternatives. Those alternatives can be, proposals can be published as draft in advance and they can be argued over. They can be amended. People could prefer this or that code version and ultimately voted upon and become part of the formal law. But anyway, it strikes me that in sort of a similar spirit to when I said one out all three, so too with just the question of what is the translation or the instantiation in code of public law, that itself, maybe a way to establish which way to do it might itself be a political question. And luckily we have methods to resolve political questions like one of them is we can elect people, they can argue, they can vote and then they can publish the answer in a law book, in this case a code law book. So anyway, those are just some musings on that. Thank you so much. What a fascinating flash talk and a great opening to the discussion. Thank you. Thanks for having me as always. And thank you, Renita, Brendan, Brian for such great questions and Daza, of course. Thank you. Okay. And so the fun doesn't stop. We actually have thanks in part because we're using the quick way to give ideas in other groups. You might still be hearing somebody kind of slogging through slides on their ideas, but no, no, we're doing flash talks. So we can get the kernel of the idea out quickly and have more time to discuss. And so we have time for a second one. And our next flash talker is TMA Rogaire and TMA is going to give us a flash talk on algorithmic sentencing. TMA, you have the floor. Thanks, Daza. Hi, everyone. Like Daza said, I'll be talking about algorithmic sentencing. And similarly to Megan, this will be a reprise from my talk during the computational law course. And specifically, I'll be talking to you guys today about how algorithmic sentencing will affect our judicial processes, particularly the trust and the standards that apply to the role of a judge as more and more countries like Estonia and Singapore start experimenting with using algorithmic sentencing in small claims courts. And even though what's happening right now in this space is going on all over the world for the sake of simplicity, we'll be keeping our discussion to the U.S. justice system. So a few years ago, a study revealed that U.S. judges disproportionately sentenced black defendants to longer punishments than their white counterparts. And while it's unsurprising that human biases to blame, finding a judicial solution proves surprisingly fraught. Although algorithmic sentencing seems like a good fix, carefully coded sentencing programs can be used to unveil judges, sometimes flawed human programming. And these sorts of disability to point out flaws should be able to help avoid situations like when an Ohio judge went against the recommendations of both the defense counsel and the state prosecutor and condemned a 55 year old black woman to 65 years in prison for nonviolent theft. The technology is still nascent and it's prone to certain issues. For example, the data use of program code is often incomplete or incorrect, which can bias outcomes. And like Megan talked about, there's ambiguous legal drafting, which in the U.S. can be intentionally left to the interpretation of judges. And so we have to be able to use our higher powers. And this complicates writing legislation that algorithms can process. But I believe that it's reasonable to assume these issues will eventually be addressed. But then we have this larger question that remains. Will algorithmic sentencing negatively impact Americans faith in the U.S. legal system? Because the efficacy of our legal system hinges on the ability to deliver justice. And while most Americans have collectively agreed to entrust the deliverance of justice to judges in spite of their shortcomings, it's unclear whether this negotiated custody will persist in light of evolving technological alternatives. So what I wanted to talk about today was to think about three things. To think about what the tradeoffs are between the current legal system in the United States and a potential future algorithmic legal system. And second, to think about whether the fairness gained through algorithms will be worse with risk relinquishing human accountability that we gained from judges. And thirdly, would a shift in an accountability from judges to algorithms, effect visibility of our currently reliable, albeit imperfect legal system. Okay. So that I'll open up the floor for our discussion. Thank you. Robot judges, anybody? How would you like your liberty to depend upon that? I guess as, as like a segue between what Megan talked about and what I just presented, I think. You know, Megan mentioned how. In the US. We deferred to our authorities, right, to help interpret our rule of law. And I think that. This question of how we want to be interpreting our laws, that that question is something that I think is a conversation that has to happen right now. So I'm interested to hear what you guys think in terms of whether or not we think algorithms are a better arbiter of justice than flawed human beings, or if there's value in a person at the end of the day, making moral judgments on our society. Okay, Renata. I think. I'm you, Renata. Hello, everyone. How's everyone? I hope everyone's fine. Thank you, TMA, for your. Amazing talk. And as Megaman. Well, I think that algorithm thing sentencing will be very valuable to those questions that were context. Isn't as relevant as a few objective facts. And I say that from a perspective of a judge who's been working with. Tax law on. For one year now. And in cases where context is not relevant. It's pretty clear to me that decisions should actually be automated by algorithm because we wouldn't have to actually use all entire powers and resources during things that are pretty much automated. But there are other situations where context is really irrelevant. And I think that in those cases. The human presence in the decision process is really relevant. And that would actually make a consistent difference. In the final justice of a, of a decision. So that's just a remark I'd like to take on that on that. Aspect of your questions. May I, may I intervene for a moment, TMA? To ask. Her honor. Maybe give us just an example or two of times in, in your administration of, of, of tax, or adjudication of tax cases where context was relevant. And where context wasn't relevant just so we can kind of get on to the same wavelength with you. Okay, sure. Okay. So here in Brazil, we've got a specific tax for. Vehicle possession. Okay. So if it's like someone who's actually delayed the tax. The payment of the tax or something like that, that wouldn't, that wouldn't be as relevant as for instance, a benefit where people with disabilities have here to actually buy their vehicles. The cheaper and. In special conditions and actually pay less tax. So. Now the law has actually changed. And there aren't those benefits anymore. And there's actually now a judicial discussion concerning this, these situations where we can, where we actually need to evaluate the nature of the disabilities, the nature of the benefit and what are, what we are actually looking forward in what we are actually trying to. Trying to reach with the decision and with the law. Thank you. Okay. Did you have any, any thoughts or remarks on that? Yeah, I was thinking this reminds me of. You know, where judicial interpretation comes in the US with due process and just thinking through, you know, how. Like there's this big scope of, I think, you know, about this a lot more than I do. And there's a lot of different ways, you know, there's like the strict deterrence intermediate. I forget the first one, but anyway, there's always been ways to interpret what constitutes due process in the United States and what our rights are constitutionally. And that interpretation. One, it differs like a very controversial. Application of due process is with affirmative action. And the fact that, and then there's, I forget which justice it was. You know, the interpretation of, of due process and how it applies to affirmative action. Not only depends on the interpretation of the, you know, written laws and how we interpret the process, but also the societal context of what's considered, you know, within the scope of normal at a certain point in time in history. And so I. Yeah, I guess thinking about just how the disability act in Brazil. And for, you know, the. I forget what it was tax tax code. Yeah, it's a tax benefit. Just like how that will shake it, whether or not like that also is a situation where it will shift with time, you know, as what constitutes as a disability changes or what the benefits ended up actually being. Yes, actually that's being discussed right now. But I think that. So concerning due process of law in the procedural laws and specifics when we're talking about actually how a process will develop inside the judicial system, we can think of. Automating and judicial sentencing and maybe having a few decisions automated, but not those where we can. Once again, where context is relevant. So here in Brazil, we've got like something that actually approaches us to common law. For instance, when we have like several similar judgments. We are obliged to actually apply those judgments. So when justice at size one thing we just have to follow it through when it reaches a feel of a few requirements. So, but for us to change that we need to take into consideration like the context and the evolution of society and things like that. And I think that maybe in those cases, algorithm decisions would actually prevent us from that discussion from overruling and distinguishing and things like that. So I think that we should be just a bit careful. Automation is really important. We need it. It's just something that we've taken too long to actually use it, but we can't forget context. Otherwise we'll just have the law will be just static. And that's not what we're looking for. Amen. You can almost imagine I'm going, I'm sorry, I'm abusing my moderator's privilege. And Brian has something to say as well. And then we've got two hands raised, but I'm here goes. I'm abusing it. It makes me think in the future, one thing you can imagine is some laws like we'll hear about next month again from Andrew when he talks about strict liability in the context of automated legal entities as a legal place thing, you know, like just like possession of us of a drug sometimes just like that's the only question. It wasn't your intent or other circumstances. Just a fact question. Did you possess it? Like, so maybe it's a question that was in my pocket. Was it on the table near me? But still it's like a yes, no possession. And some of these can be simplified by the legislature making rules that are very, you know, kind of perfect for algorithms. Others like due process and civil liberties and living concepts of justice, not so much. On the other side of the coin is litigants. And so you could imagine even with something that theoretically has, you know, or had a clear interpretation like possession in one situation in the future when things are digital, maybe this is now up for a context debate again. And so you can almost imagine a new class of civil or criminal procedure hearings like a context hearing like your honor, I object, I require a context hearing. Okay. Do your filings as to why you believe context matters. And then you have it out with the other side and then there's a ruling whether context matters or not. But anyway, this context thing I think is just, I'm so glad that we brought this up. I think that we're surfacing something that's going to become a key feature of computational law. So with that, I believe we have some, some new moderator privileges from Brian. Yeah. So I, one of the things I've got one point and then kind of an open ended question that I want to pose. And the point is that when you think about the efficacy of. You know, something that's fully automated versus something that's analog. I think it's easy to get caught in this false binary of those are the two choices. I think there's a middle path where it's like the augmented. And I know we talk a lot about, you know, not being C three P O, but being in the Ironman suit where you're the thing that you were trying to do is augmented by the technology and not replaced or, or eliminated by the technology. And there have been case studies that have shown with, for example, certain types of legal work that, you know, if you train a contract system like log gigs did to review non disclosure agreements, you can automate and find, you know, 95% of the issues that might arise, which is better than, you know, if you allow humans to do it and only get like 88% of the issues that might arise. But when you use the two together and you combine the two together, you get 97, 98% of the issues that arise, which is higher than each of those alone individually. And so the question that I want to pose is if we begin to think about this concept of automated justices or automated sentencing as one that is more of a, you know, how do we supplement the abilities of judges? How do we augment the abilities of judges in a way that empowers them to, you know, get to closer to that 98% instead of the 88% of the 92%. You know, what does that begin to look like? And I don't know that I have an answer. I don't know. That's the end of my question. I think you have to now give like the factual, definite 100% answer. Well, I, so I'll throw a link into the chat. But I think one beginning step, something that would be an easy thing to do would be like what Gabe Tenenbaum and James and Dimitri. Proposed in their article that they published with us. May it please the bot, which is machine readable judicial opinions that. Allow us to at least to begin to contextualize some of these decisions that are being made in a way that would, you know, support applications built on top of them. Any thoughts or reflections on that? And we do have two more that I think we can squeeze in from Bruna and Brendan, but. Okay. So we'll let your contribution stand as its own. Contribution Brian for now. And Bruna. You, you have your hand raised. You may unmute. Hey. Hi, everyone. So I actually have a question. That would be that is aligned with what Brian just posed and gave an example, because when I, when I think about on automating. And then seeing out the meeting out the nation. I actually also think about the, the first tax that was, was, was assigned for the computational law workshop for, on algorithm. Let me just make the name a perspective on algorithm. Only our algorithms. And I, I think when I think of the missing components of the. Current legal production. I think, yeah. Those steps are definitely desirable. But I have the same questions about. How it is assigned for the computational law workshop for on algorithm. And I also have some questions about how it is, how can we actually make on. This hybrid. If we're not to, if we're not too. Sorry, I'm just trying to, to, to formulate in my head. If we're not to. Replace if we're not just replace humans. And especially, and also in the metrification, for example, and evaluation if the goals are being. Fulfilled in a, in a, in a legal algorithm. So if we're not to replace and if we are to augment. Is there like an example of an effective on hyper systems, because when I think, for example, in hanging systems that could be used. I think those. I think the, the, the evaluator, the, the people who is making the decision is not going to, to check all the, all the, the final, all the pages and all the options, for example, and they are going to, to actually just see the first ones, because they are relying on the, they are confident about the computer ranking decisions or something. So I think I have a serious, serious question. And I always get really anxiety about that when I think about effective hybrid systems. And since I don't have that many experience, I would like to hear if there is an example of a contextual application of on legal algorithm or sentence and algorithm that is effectively on hybrid and not just sick. It says it should be on hybrid systems, but it relies on the, on the computer after all. Thank you. If someone has an answer, I should have a quick comment to add to the, for any answers are provided. Please. What. Yep. Please do. It's your show. So I oftentimes thinks about, think about the tendency for human, in human close human systems for entropy to happen. And I think that what I worry about oftentimes is in an ideal world, we would use, you know, algorithms as a tool to augment our decision making. But because of this, you know, because of entropy, we tend to just default, you know, I have a, an email program that I use that helps sort my emails for me. And rather than actually going through and tagging things, I've just defaulted to just trusting whatever the heck goes through the filters and it's going to tell me something spam. I believe it. And I've discovered recently that that isn't always the case and things get thrown into the wrong bucket all the time. And that this is something that's really not that important right. My emails, but I think that I sometimes think about what, what happens. Yeah. Like what would happen in a judicial system. And I also, another point that I'll bring up is, you know, in the U S there's been a lot of reason, a lot of recent more recent studies into how our judiciary even works and like the decision making process that happens. And it's sort of highlighted that judges are humans, right? Judges think very critically and like they're, they go through a lot of schooling and just what they do is incredible, but they're still humans and the stakes happen. And rather than having these studies point out, oh, here, here are some places that we can augment judicial like human decision making with programs. What ended up happening is a sort of decay of trust in, in judges. And I think that that's, that dichotomy almost between like this potential for making the entire system better and this potential for sort of completely decaying the trust that we have in a system and just collapsing our system is just something I, yeah, I think about a lot and I don't really have a clear answer for how we can improve it, but just know that, you know, debates might help highlight, you know, our through idea flow highlights some, yeah, potential solutions. Okay. Anyway, Brian, I know you get something to say and Brendan too. So I think that was a really good kind of like framing of where we could go with a lot of these, you know, with this augmentation because certainly, you know, there's something that is lost in this translation process, like tying it all back together. You know, there's something that's lost and if, if we just completely rely on translations and did one example that is easy to point to is if you look at Google translate, you know, it relies on context that is, you know, completely divorced from, you know, whatever the larger situation that you have is and that ties back into some of the work that Douglas Hofstadter's done on machine translation. And then, you know, I think, you know, losing the, we have to make it so that the goal is elevating the human and, you know, just reducing the potential abilities. And that's all I'll say so that we can move forward. Brendan, you're up next. Oh, great. Tammy, just a wonderful discussion. It's all very important. And while you're focused specifically on real space and, and meet space for lack of a word, is really, really important discussion for all things made out of atoms and also all things made out of bits. And to carry further on what Brian is talking about, the nail that hits, or the hammer that hits the nail on the head is the idea of context. And where I think your work is really important is exactly in this and the constructs of, and it's been mentioned before, simulation. And where what you're doing and how you're approaching things, how that meets the world of simulation. And beyond being able to simulate things, another very important part of this context switching is the world of oracles. And this is something that is going to come up by and large, I think specifically in the Wyoming jurisdiction in terms of their quarter of chancellor, because we have this situation where we have automated smart contracts. And then the question is, you know, at what point does that break down? At what point do we actually seek to have human intervention and interpretation? So we have the case that we have two different groups, two different systems, two different areas of research. One is from the world of atoms and one is from the world of bits and smart contracts. And they're kind of coming together and where they meet in the middle is this transition of or questions of how do you do the handoff in terms of reinterpreting the law in a different, I don't say systematic context, right? Going from algorithms and then switching over and saying, okay, this is not working for what we're doing here. We now need to switch over to oracles or human intervention. I'll leave it at that. Thank you so much. That's actually, I love that framing of it and it's useful to think about it in that way. Okay. Thank you everyone. I'll hand it back to Gaza, but I really appreciate your comments. It was really incredible. Okay. Thank you so much. Yes. We truly do live in the worlds of atoms and bits at the smallest increment of matter and information. And so thank you very much everybody to our speakers and Brian and to of course all of you in our growing community of members and of idea crafters. And so we hope that you will join us next month on the last Friday at 12 p.m. Eastern for the next episode of idea flow. And we'll have among other things the flash talk from Andrew on automated and autonomous legal entities and we'll have a second flash talk. And we have some really interesting people in the pipeline for May and June. And if you have any suggestions for speakers or a few yourself would like to give a flash talk, please feel free to to let us know. There is a form for this that you can also share with others. We want it. I wanted to keep it to people that had been in the class this first time just to get our flow. I believe we have our flow. And so we're going to open the doors in the future. That form is at computational law.org. And that's on GitHub. There may be a secure. I need to hook up the SSL certificate. So you might get a terrifying security message right now, but that's where the Google form is anyway. And so thanks again, everybody. And we look forward to seeing you next month on idea flow. Until then, we wish you well. Bye.