 Good morning good afternoon or good evening wherever you may be in the world. My name is David Horrigan and this is the law dot mit.edu podcast with Daza Greenwood and me. The law dot mit.edu is a podcast that explores the rapidly changing worlds of law and technology as our lives become increasingly more digitized. It's important, more important than ever to start thinking about reimagining and reengineering the law and how it interacts with technology. Each episode we're joined by thought leaders from around the world in order to demystify and break down the complex topics at the intersection of law and technology and help you better tackle the topics. Now of course a gentleman who needs no introduction, our partner here at the law dot mit.edu is in fact the executive director. Our good friend, Daza Greenwood. Daza, what's up for today and what's our topic. Oh, thank you so much, David. And, you know, the topic today is one near and dear to my heart and actually that is captured the focus of the entire law dot mit.edu research team and also our publication MIT computational law report. And that is none other than generative AI for law and we're going to do a deep dive into one of the key implications of what does it mean to successfully use this technology when you're practicing law, but you know, I don't want any spoilers yet at this point we're just introducing ourselves. And so that's who I am. That's what I'm doing here. And Dave, take us to the next step. All right, thank you, Daza. And to say that we have a consummate expert with us today on this subject would be a drastic understatement. Our guest today is Olga Mack, and many of you in the world of legal technology know Olga. Olga V. Mack is a vice president at Lexis Nexus where she serves as vice president and CEO of council link CLM, formerly known as parley pro Olga was the CEO of parley pro a contract life cycle management platform which was acquired by Lexis Nexus in 2022. And here's where we get into a true Renaissance legal technologist. In addition to her work, bringing technology to legal teams Olga's an accomplished attorney, having served as commissioner on the California Commission Commission and as the general counsel of the SAS firm clear side, as well as the general counsel of the Pacific Art League, counsel at visa, and counsel at Zeus, and we'll get more into that later on the broadcast because we'll find some interesting connections to the world of legal technology. Earlier in a career she worked in big law at Wilson San Sine. And if that weren't enough. She's a former assistant district attorney in San Francisco Olga Mack. Welcome to the law dot MIT dot edu podcast. Well hello everyone David good to see you, Daza. I love the jacket. Amazing you didn't have to dress for me but look amazing. I'm really excited about this conversation. And we are to and we're just disappointed that we cannot share Daza's vibrant jacket with all of you because this is podcast land, but let us just say that Daza is looking terrific and very spiffy today. Hi Barbie. Hi Ken. Hi Daza. It is bright pink. It is beautiful. It's a shade of pink for the listeners and Daza in the jacket shade of pink is a beautiful site. No doubt about that. Alright, we're going to start off today's broadcast with a little bit of a quiz. And I'm going to ask Olga and Daza, if they have heard or have any familiarity with this with some of these cases. We have a Shaboon v Egypt air a 2013 decision out of the first first appellate district in Illinois. We've got Peterson v Iran air from the district of the District of Columbia a federal decision from 2012. And as the Delta Airlines, we're getting a theme here of something in the air. It's a Texas appellate decision from 2019, a state of dirt and v KLM Royal Dutch Airlines, a Georgia State Court case from 2017, or perhaps early the American Airlines from the appellate division in New Jersey 2003. I'm wondering about all these cases seem familiar to you at all. Have you ever seen them in a law book, perhaps. Wow. Yeah, I think I, these are, these are like the dream cases that I most want to site in a brief. Wow. Perhaps. Yeah, it's exactly like these are the cases that chachi PT shovel to the, that lawyer that got in trouble in New York right. And as I get this one you are correct sir these are the fake citations from Mada the avianca ink. And we're going to get Olga's expert opinion on some of the things that come up in this case one of the big things is supervision and I think a lot of you have heard about this case but we'll just give you some brief background. Mr Mada was traveling on an avianca air flight from El Salvador to JFK. And, you know, I'm six foot for we have had a lot of tall people and sometimes if you're really tall you're cramped in that seat, your leg may be in the aisle. Well the drink cart comes down the aisle and allegedly once again facts are not decided yet but allegedly the court is thinking something seems odd here. And specifically it was one of the 11th circuit citations that tripped them up. And so the court actually called down to the 11th circuit was like, Hey, have you ever heard of this case? And specifically it was one of the 11th circuit citations that tripped them up. And so the court actually called down to the 11th circuit was like, Hey, have you ever heard of this case, and they're like, No. Well, you know where this goes. One of the attorneys that actually just asked chat GPT for some legal citations and chat GPT spat them all out and they were all fake. They were sanctions involved in this case. And the supervision issue is on this the case was originally a state court case it was removed to federal court, and the attorney signing and giving the affirmation on the ones was not the one doing the legal research. He was the one who was admitted to practice in the Southern District in New York and federal court. So, his legal colleague was the one who was doing the research with our good robot friend chat GPT. Never bothered to check I'm thinking well if it's chat GPT it must be correct. So Olga Mack we've got issues of generative AI we've got issues of the supervision of attorneys. We think about our model rules of professional conduct rule 5.1 on the responsibilities of a partner and supervisory lawyer rule 5.2 responsibilities of a subordinate lawyer, but so we've got as I so often don't don't forget rule 11. Oh, yes does a, I would be negligent on that one so we've got, we've got all these issues, plus generative AI thrown on top. What's your first takeaway Olga. Wow. There's a lot of takes away. As we know, hallucinations in the real problem with generative AI, or this hallucinations generally this technology is designed to please us. And sometimes what pleases us is doesn't exist in the world, you know, I've ever been married had relationship with anyone, you know, a good, well intended compliment that may not necessarily correspond to reality improves marriage. But we have a good relationship for that matter. And, and chat GPT pleases us it knows we're looking for something it really wants to give it to us. And then it does. Who cares that it doesn't exist wait, the court cares. The court actually cares that it doesn't exist. And that's inconvenient. So, my first reaction is we either need to solve the hallucination problem or we really need to fix our courts, because they just raining on our parades when we hear what we want to hear, and jokes aside, I mean, listen, anytime you use, you do anything as a lawyer, it's your job to make sure that this stuff is good when you do a filing in court, you put your signature on it, whether it's because you appear or some you appear in somebody else's behavior, how you are the holder of your license, that means you need to be comfortable in whatever representations are made, whether they're factual, or actual law representation in this case, clearly, if all this case law doesn't exist, it only exists because it pleases you that's while satisfying at some level, not satisfying to the court. And so, yeah, so many failures of competence, supervision, all kinds of things. And that just means really, in the end of the day, low technology companies, which leads to, you know, no practice essentially, and inappropriate use of your license and putting your economic ability to earn living by practicing law. A jeopardy. Sure. Okay. You know legal tech as well as anyone you were a 2020 fast case 50 award winner. You received the women of legal tech in 2022 from the American Bar Association, you know this stuff. And it's often been said that technology can be the great equalizer for legal teams, allowing the small firms to compete with the big ones. You know, the general counsel for smaller organizations, you've also practiced in big law. What do you think we have a firm that got into trouble on this one, but do you feel that technology, and let's go a step further say genitor, gen, generative AI can can level the playing field from the small firms and the big firms. Look, I think technology is an opportunity for us to redesign what we find unsatisfying with the world today. I think technology gives us a restart. Just kind of like what you have with the computer every time it does wonky things, you push, you basically restart it. I think that's what technology allows us to do. There's a lot of things that are unfair and just, you know, including the fact that, you know, smaller players don't always have an opportunity to have an impact, for example smaller law firms, and I absolutely agree that something like generative AI technology can be a representative player or smaller player or newcomer. I sit at the table, you know, I shot it in the game. With that comes also an implicit understanding that you need to understand the thing that helps you with that right comes a massive responsibility, because technology and I feel like I say it all the time now is a sharp knife. It could be used to cure, and it could be used to kill. And the difference is going to be that delta is going to be a knowledgeable operator, a person or or a collection of people are more likely to do good with a sharp tool. If they are trained operators who have an analytical framework of getting from point A to point B, have a definition of success, and who possibly have done it before. And if you have none of that, then you are lost. And you may be misusing the tool, you may be in danger to yourself in danger to others, and you can be with this tool in danger to society at large, because this this tool actually allows you to mislead, for example, at scale. And that could be very damaging. That's why this tool specific with the RNA AI could solve a lot of things, but could also be that that knife that kills. Great, great analogy as unfortunate as the end result may be. Let's take it a step further and does it be great to get you in on this as well. A lot has been said about the invasion of the robots we've talked about the level of the playing field, but, and we talked about, maybe it can level the playing field, but is there really the concern that is going to take people's jobs we have the strike right now going on with the writers and the actors in Hollywood. We have a lot of people in the legal community concerned about boys is going to take all our jobs. Do you think both of you do you think it's more of a tool to help. Or do you think there's a legitimate concern that we're all going to be working for the robots pretty soon or not have any jobs. Wow. Well, I did want to pop in on that and also to maybe deepen the question a little bit for for Olga, but to me just the framing around the the matter case. It's so intriguing because and the reason I mentioned rule 11 is the fundamental obligation is to practice law competently which means you know checking sites, ensuring that the legal arguments are in the best interests of your client, etc, etc. So rule 11 codifies that obligation for filings or representations to a court, you know that among other things that you you've done your job that some due diligence, you have an evidentiary basis for the assertions that you're making in the first instance. But I think this is so fascinating precisely because Well, so just on that one, I'll stick on that for one second and then I want to really talk about supervision primarily, but to me that seems like that's another way of saying, there's a, there's a critical role for a human with respect to the use of generative which is competently expertly diligently reviewing assessing evaluating, you know critiquing and correcting or outright rejecting the outputs of generative AI that that is the essence of practicing law. And it's so interesting because of the duty of supervision, I feel like plays out in multiple ways here in terms of like the lawyer job preservation, like active 2024 and beyond. It seems like it's not merely supervising or I guess it's a question is it just supervising people who are using generative AI or does the duty of supervision is have its own valence with respect to supervising the tool and the process itself so I, I guess I'd love to talk more about that but all of this whoever is doing the due diligence in the law practice, whoever is doing the supervision. These are all human beings. And so I do feel like we're not replacing humans where we're super charging humans with the technology, and we're further honing and applying the duties in light of the technology, but that raises these open questions it's new technology, how do we apply them and like and I'd love to dig much more deeply into, into this human role of supervision and how that plays out in the, in all the various ways that this technology is going to be used. What do you think Olga. You know, look, duty of supervision. I mean, I've done recently quite a lot of research and thinking about it. A couple things. It's, it's not unique to lawyers. First of all, other professions have it either codified or as a matter of practice. Observation number one, observation number two, it is also not unique to United States lawyers. It pretty much one way or another exist in most democracies and most places where you see lawyers. So it's a pretty universal beauty really one way or another. So what I'm going to show you through what I find is that what it means is actually a point in time question. It started sort of with a sort of traditional apprenticeship model which would you use to exist in every part of human society from artists to lawyers to publishers to whatever. It sort of started today in includes things like ensuring work quality, promoting ethical conduct, providing training, managing cases, maintaining client community that includes all kinds of stuff. But it changed over time it changed over time from something you watch me do it and then you can do the same. Another thing more like let me show you best practices here is how to deal with privacy issues and ethical concerns. And I think where it is evolving into something much more comprehensive that includes things like supervising technology, we're moving away from supervision meaning you trained to be a lawyer because we now have law schools really, and we have bar exam but kind of like takes care of a lot of this early apprenticeship stuff. Now it's sort of in the beginning of your career just to make sure that you like, don't get, you know, you don't sink in the pool completely. But it involved into something more of sort of helping you coach you through career doing the right thing and thinking through ethics and then not just a lawyer who is who just graduated from law school, but most of me who members, because their results could be input in your filings and you are certifying that you supervise them, for example. So it involved into much more than apprenticeship model to much more comprehensive sort of general duty of supervision and all kinds of stuff. And I think where this is all going is that supervising technology to and we've been basically certifying with the discovery for example, it's already been there. But now we have what you know what David described as Rob robots is here and we may be working for them. I'm not sure this is where we're going robots will still be working for us for quite a while and it will be our duty, according to model rules to supervise and I'm pretty sure that is not a stretch of the rules we have today. And I think other professions will be supervising and codifying or at least developing best practices about how to do it. I'm going to stop here because I can talk about it all day, but in a nutshell that those are the three trends that I see. That's great. By the way, and we may not be working for the robots because if the recent congressional testimony means anything we may be working for the Martian soon. If you've heard about that and this is not some inebriated guy in a Cessna. These are long term United States military officers testifying that. Yeah, you know there was biological material they brought them in so we may have bigger things to worry about than the generative AI robots but it does. It was great that you brought up rule 11. Because the court of course sites rule 11 in moda via Bianca and specifically rule 11 C one, providing that absent exceptional circumstances, a law firm must be held jointly responsible for a law for a violation committed by its associates or authorities. And we talked about the model rules of professional conduct 5.1 and 5.2 and 5.1 is basically I'm going to paraphrase it but it's basically saying hey partner supervisory lawyers, you're responsible for what your lawyer is doing and 5.2 goes on from there, saying that, you know, the subordinate lawyers, just because your boss told you to do an illegal thing doesn't mean you're off the hook you've got to follow the rules instead of that. So when you're thinking about the rules and thinking about this situation. Do the lawyers think about what they could do with generative AI. I have to admit, if I were the supervisory attorney in this case, even though the world changed a lot on November 30th the last year when generative AI really hit the, hit the whole nation, the world, etc. I'm not able to think that someone may have gone and checked that I wouldn't have thought, gee, I need to ask him, did he use generative AI to do this. Is it now I mean a lot of people have heard of it, but how does AI change the rules of the ballgame here. Do we need to now add it into the federal rules of civil procedure that there are rules on using chat GPT for surveillance. I mean, at the White House recently, some of the leading tech CEOs gathered with the president to talk about voluntary guidelines. Do we need to put generative AI into the rules. Olga, what do you think. The technology it's been in SAS data AI ML blockchain crypto numerous under disruptive technology before they were cool and earn their buzz. I know the knee jerk reaction is to regulate technology. But really laws regulate behaviors. Sometimes when it's hard to regulate behavior, they, they kind of focus on impact and outcomes. That's sort of our standard way to write laws, I kind of think we should speak to that. I think it works general speaking. I think the rule of supervision is there. It is clear enough. It has extended over time, as I just talked about. I don't think it's a massive stretch to extend it to supervision of machines, whatever shape form they come into our lives. What's that is not to say that we're all great. So I think there are pieces missing. Here's the pieces that are missing. What's missing is the how what's missing is the analytical structure. What's missing with definition of success. That's what's missing. And I can kind of maybe talk a little bit in greater depth about it. The rule is pretty simple. And that applies to human. I guarantee you, if you serve a 2100, 2000 lawyers or professionals, want to become professionals. There will have different approaches to supervision, different definitions of success, and different ways they deal with shortcomings of their individual supervising models. We do not that definition, we do not teach law students. What it means to be a good supervisor. I, I had once been assigned a gentleman to be my mentor. After he ignored me for about three months, we ended up in the same elevator this gentleman happened to be always very well dressed. As we were going down in the elevator, this gentleman was a lawyer. From a 30 years forward to the first. We had a brief exchange where he said, Oh, I realize I need to be a mentor. And let me give you practical advice. When you travel, make sure you bring an extra suit, because sometimes you spill stuff all over yourself and you need to change. I'm sure he meant well, and this is why it sounds solid. And in his mind, he checked the box of mentoring. I can tell you as a junior lawyer, I got off that elevator confused, confused whether he thinks my suit was not up to par whether he thinks I'm a slug, because I'm likely to spill stuff all over myself whether he thinks I don't know how to pack. And generally, well that the excitement about this senior lawyer mentoring me would extend to something more exciting like how I practice law. But in his mind, this was very practical and useful and he meant well. I think we're both right. We just operate from a different intellectual space of what it means to be a supervisor and a mentor. And what's helpful to a junior lawyer. I think it's an extreme example, right. But if you know people do not have a shared definition of success of supervision, they have no analytical framework or what it means to how to do it. And they absolutely don't have a way to measure whether they're doing a good job. That's what's people that's before we get to machines. If you add AI on top of the fact that we already don't know what we're doing. Then it's hard. The problem is not the law, it's operationalizing the next steps envisioning success and measuring results. Yeah, you know, with I do a lot of mentoring as part of the role at MIT with younger students. And it's so important to tune the mentoring to the mentee to the person you're talking to, and to really understand them. I'm sorry that happened to you. It reminds me a little bit of when I was a very young lawyer. I got kind of thrown the deep end very quickly in a state government and with respect to technology and end up testifying the first time I testified in Congress. I was young, like in my twenties and I had my suit I kind of flew down to DC. The clockwork dumped a bunch of horrible things over breakfast on my suit. I couldn't wear the jacket. I did have an extra shirt, which I put on and then the staff wouldn't allow me to get onto the Senate committee room, not wearing a jacket I had a tie, but they're like you can't do it I think it's relaxed. It was not finding a jacket for me that was like I don't know five sizes too small look like one of those monkeys with the crane or something was horrible but you know, in my case as someone who apparently was a slot like that actually could have been good advice to me but you have to tune the advice to the person, it has to be meaningful you can't just check a box and, and you're right that we have so far to go with suit with what does it mean to mentor where's supervision mean. How can we make it effective how can we rely upon it when so many people don't know what it is how to measure it or how to do it it's, it's a whole new horizon. That's that's with humans. Right and with humans you can, you know, at least I give you your EQ is a little higher you can say he doesn't tell me where you're in life. One of the biggest challenges, how can I be well that's easily sold. And I do think the gentleman meant well it would be helpful in some circumstances. And, but with machine, you really need to have an approach you need to have a definition of success and you need to be able to deal with said that. And today, we basically telling lawyers you have, you have a rule, you have a duty. Do it. Do a good job. If you fail. You may be disbarred in your economic right to make a living as a lawyer that you work so hard, maybe lost. Good luck. May the odds be in your favor. Which brings us to what you said in the end of your statement is what's missing is how we have the rules the rules are pretty well adapted are capable of being applied to different circumstances and new technologies, but we're not born knowing this stuff the technology is new. And I feel like I'd be remiss if I didn't mention that Olga is also an esteemed member of the law dot MIT edu task force on the use of generative AI for law practice. Which is recently published principles and guidelines to hope we hope help to begin to put meat in the bones for how to apply our existing rules of professional responsibility to the use of generative AI and one of those rules is I think the principle seven is supervision. And but we're just scratching the, the tip of the iceberg here in terms of how to do it and really there are going to need to be rubrics and practices and you know examples that are, here's a good example of how to do it here's a bad example of how to do it like, we've got there's a big open space there and half of using generative AI is getting the prompt right. The other half or maybe 60 7090% is is evaluating properly the outputs before working it in, and that that really is the essence of the supervision isn't it. That rule needs to be operational that's that's business fee. We need to take the rule and create a framework, and then based on that, through examples and checklists allow folks to do a good job and measure success and have a shared definition of success. And we that that's a standard operation procedure in any complex task and make no mistake, supervising anything or anyone is a complex task. It requires a framework it requires a checklist, and it requires a vision of success. And so I think that's the part where the white needs to be filled to answer David's original question. I don't I think we have enough laws in the book that doing the right thing. I'm not committing fraud, not lying stealing cheating like the basic things that why it really just sort of encouraging what I call civil society but people call civil society and good behaviors, we have enough of that in the books. You don't have to regulate technology you regulate behaviors, you can extend much of what we already have to any technology. What's missing is analytical framework sometimes this is where trade organizations like a be able to be helpful, or maybe regulatory organizations could be helpful, or professionals getting together could be helpful, and sort of development of examples scenarios, best practices checklist checklist that key. Those things need to be happening with supervision for humans or machines. There's a game going on today of do your homework, and how supervision comes in different ways you know you were telling stories from way back when, when I was much younger we were working on a case, and we were in what dear and doing the jury selection. We got back, and we're in the room conferring one of the perspective jurors had worked with opposing counsel. And so I'm the young guy there and I pipe up. Well, we've got to strike her they used to work together, and the senior partner goes, Nope, we are not going to waste a strike on her. And he could see I had this quizzical look on my face. And he goes, that guy is a bleep. That's not what he said. He said, if she worked with them. She's got to hate him. They'll strike him. We're not good they'll strike or we're not going to waste a strike. Sure enough, that's exactly what they did. And so I was the young kid who didn't know any of that stuff, but it equates to what we're talking about in supervision comes in many, many ways and it's knowing the people and sometimes generative AI is not going to know the people and know the case law. But on this deal about different ways of supervising Olga you are the quintessential American lawyer you're a graduate of the University of California Berkeley School of Law member of the California bar. You work for major American law firms, but in your work as a general counsel here in the United States. You've also had big international experience some of these are international companies doing international business. And in fact, you emigrated here from Ukraine at a very young age so you are not having tunnel vision in the US. When you think about this supervision. Do you find that differs in different parts of the world. So I've done a lot of research and I definitely as somebody who was born elsewhere in English is my second language and I travel quite a lot, and have been in companies with massive footprints of United States. I have a sort of my sort of to this rule of law concept they have a much worldly presence. And what you find is that the duty of supervision is actually pretty universal definitely across democracies. And where you see high concentration of lawyers, which is like Asia, EU, Brazil, places like that. You definitely have some sort of codified way of duty of supervision. And they all emphasize competence as a critical aspect of supervision. They emphasize supervisory responsibility such as work quality, ethical conduct, oversize all of that is sort of universal recognize client service and acting in clients best interest those are really common themes. There is sort of a very, very degree of specificity and details, and they're definitely enough differences but but I would say to me overwhelmingly those things are have a lot in common. Some of the countries include sort of the themes around business management others don't. And some have sort of more specific guidelines and some have some more regional EU, if you practice it if you know, you as a member country in the United States in the in the European Union, you may belong to the equivalent of the bar there and maybe a new lawyer as well. So you may have overlapping duties of supervision. So I would say it's close enough and most of your diction there are differences and in specialist specificity, but it is definitely there it is common theme and I would go as far as saying that they more or less evolve over time and lock steps, sort of expectations of broadening that duty to include vendors to include ethics to include technology, all of that I think is very much rising around the same level across regions. Olga one thing that we like to do on the law.mit.edu podcast is get to know our guests now. We've known you professionally for many years, but our audience may not be familiar with some parts of your career journey. And one of the things that we found interesting was your work as an assistant general counsel at Zeus. And for those of you who may not be familiar with it. It is an online dating service operating in 80 nations all over the world. There are 25 different languages. It seems to be a great collection of many of the issues you've got online you've got data, you I assume have personal information. What was it like being the lawyer for the online dating service. It's a joke that I should have retired after working at Zeus, because I think I picked as an interesting person then I can tell you when I would show up to parties and declare that I'm a lawyer, I was a number two where I was an associate general counsel there that I'm a lawyer for online dating company. Most people in the room, I would, I would say all but that's probably an overstatement would find me fascinating and anxious. I would come to me and ask me about advice and, and what it's like, and things I'm willing to share and things I'm not. I was like, literally the most interesting person in the room, I can tell you that joining a SaaS company was, was a massive downgrade. And a legal tag did not improve my odds. And, and so it was a really, really fascinating place to work, because it's really something, you know, I find that love is a relatable thing to to everyone we all one way or another had an adventure in love, whether it's, it's, you know, successful adventure or the one that was a lesson, or the one that allowed us to grow whatever that thing was, we all felt it deeply and strongly and so planning a lawyer who helps people fall in love is a deeply fascinating thing. That particular company was very interesting for many reasons it's definitely not the one and only company today but at the time it was one of the first online dating companies on the internet so it's a we're talking about early days of internet. And the two founders and the staff they hired were innovators, truly innovators and yes, love, but let's break down love into into something more scientific and make it completely uninteresting. Let's talk about data, because online dating companies is a consumer facing business and online dating company upgrades across geography. And for online dating company to be successful you have to basically rely on data one way or another for at least two things. One, marketing, because that's how you acquire users and to product development because you need to figure out, you know what makes people fall in love and be interested and go on the first day and ultimately, you know, figure out a way to make that relationship successful. So, you know, behind the heart, there's data. And to me that business you take all the sort of jokes aside was very interesting, because I see any online dating company or including Zeus as essentially a data company we had a lot of data scientists on the team I was advising various uses of data. At the time where CCPA was developing and GDPR was in its infancy and it's actually not all that different from the time we have today with AI. When you have avoid in laws and practice and best practices, you see you have a delta between law and you have a delta between law and the right thing to do. And depending who you talk to that delta could be big or small and your job as in house lawyer risk professional any kind is really to have sophisticated conversations about that. Because what you know for sure is that the law will be filled with increasing number of regulations. And that may take three, five, 10 years, depending on industry. And at some point, you will be judge inside 2020. And so you can take an optimistic view and say, in the absence of anything, let's, let's do the minimal let's do the floor, because there is a void, or you can be an enlightened leader, you really are faced with that choice. You can, you can talk about the ceiling. You can talk about what is the right thing to do. What kind of world you and your children should be living in. If somebody handled data like this for you, how would you want them to behave. With that look good on internet or newspaper, all of those conversation and really kind of figure out what the delta is, and whether how close to the ceiling or for what is the consciousness of your organization. And I really enjoyed that job, because for that reason, in the context of a dating company, not only is it a data company. It's a data company that collects a lot of information, and we're not just talking about names, like cards, numbers, locations. We're talking about intimate things that you may not even share with people in your life that are your close like your parents. For example, it's not unusual for online dating company to collect whether you have STDs, your sexual orientation. All kinds of stuff that not on your LinkedIn that your parents may or may not know. Hey, your spouse may or may not know. All kinds of stuff that is very just, just private. And so, it was really interesting for me as I consider myself a privacy professional. To me that's being part of technology lawyer, and thinking through the issues of doing the right thing through the ethics of it was very interesting and we are facing exactly the same challenges today with AI. While would you have enough was on the books. Things like ethical considerations thinking about an impact of what you're developing in AI those are not codified just yet. And so many companies developing AI today are forced to think about what is the right thing to do, which I think you know David mentioning that open AI and other players went to Congress and asking for it. That's one approach to do it. But also, you can engage in that exercise was company console. I guarantee you, you may not get everything exactly right. That's going to happen in the future, but I guarantee you that if you get technologies lawyers business professionals in the room for competent and who have consciousness and want to build a good world. I guarantee you, you will end up with really good outcome, maybe not exactly right, but pretty well. And that's going to happen for lawyers with a conscience, as well as experience on the much more entertaining world of being a lawyer to help people fall in love. Olga V Mack vice president and CEO of counseling CLM Renaissance lawyer, former law enforcement official former general counsel of a couple of firms. It has been a real pleasure having you here today and thank you for your insights on generative AI. And as this thing concludes, I hope you'll come back and join us. And Daza, as usual, would you like to close this out. I would love to and I want to second that thank you so much for for sharing your time so generously and your insights and admonitions Olga and you know it's, it's, this is what makes David and I and law. You want to do the podcast as kind of substantive. And that's a entertaining programming. And so we hope that you will join us again for our next episode of this podcast and until then, we look forward to seeing you at law dot MIT dot edu