 chat box and legal expert systems presented by Tom Martin We give a brief introduction. Tom is the founder of Lawdroid. Lawdroid is a Facebook messenger app designed to help entrepreneurs solve their legal problems Basically, it helps you assist you All the necessary forms Through our expert system to fill out a future LLC in California Tom is a Yale philosophy major lawyer Techno felt Billy up to the newer It's also the managing attorney of foresight legal an ARAG network attorney and chair of the educational subcommittee of the group legal services association an ABA affiliated organization Tom is also the co-founder of the Vancouver legal hackers and if there's anything I miss Tom you can fill it in Yeah, thank you, Miguel. I'm I'm really thrilled to be presenting today and Yeah, it's a lot of stuff. There's a lot of stuff being juggled every day But I think one of you know one of the best things that I'm a part of is actually your program because it really involves the the future, you know and being able to connect with Law students and give them some insight into My experience and what I think the future holds for us Is something that can really help them moving forward I Meet with a lot of Established lawyers. I just got back from a conference for group legal services association that was held in Scottsdale, Arizona and You know I gave a presentation there about the future of legal services and You know when you get a lot of questions you get a lot of Concerns from the the more established lawyers about how this will affect their existing business and So I think by giving Law students this preparation it really gives them a huge Advantage to be able to kind of leapfrog into success Versus more established lawyers so anyways With that I'll thank you for Yeah, you're welcome. I think you're doing a great thing here And I'm proud to be an advisor With you guys so with that. Let's let's look at the agenda You know having this many things on the agenda where it's not going to be Too slow-paced, but if you have any questions, please let me know what those are and I Believe with one of these tabs I can see them or if Miguel if you could let me know what Somewhere that stand out. I'd be happy to answer them either at the end or during or However, we'd like to do it. All right, so the first Is don't believe the hype there is an incredible amount of hype right now around AI that this for examples AI that we've come So far that we have Android-like, you know robots that are That exist or that this is AI or We're here hopefully not, you know that we're gonna have some dystopian future where AI takes over. I Wish it was this But we're not quite there yet And there's a lot of myths about AI right now With all these news articles coming out and people talking about it, you know But it's intelligent that it could actually have some general intelligence about it It doesn't yet. We don't have a general intelligence AI yet That it's conscious or that we have strong AI Or that AI can be defined as any one thing So what is artificial intelligence? Well intelligence is hard to define but as far back as 1950 the godfather of computers and artificial intelligence Alan Turing he defined artificial intelligence in a practical way what he said was if you could have if you could hold a five-minute conversation by text with someone and Through that five-minute conversation You couldn't tell if it was a machine or a real person That the machine had passed the intelligence test and that test of course is now called the Turing test As a philosophy major I point you back to 1637 when Renee Descartes with his discourse on method Basically proposed the same test which is kind of strange coming from almost 500 years Well 400 years ago plus Basically his philosophy was based on skepticism and he said that there could be machines in the future That would be indistinguishable from people so how would you be able to tell the difference? And so what he said was well of course you'd be able to tell the difference because whatever they would have to say It just wouldn't connect with reality. It would sound you know kind of off It wouldn't sound like a real person which is essentially the Turing test that he anticipated the two Computer scientists that actually gave us the phrase artificial intelligence or John McCarthy and Marvin Minsky From Dartmouth. Well, they had their first conference if you could believe it on artificial intelligence in 1956 at Dartmouth and they define artificial intelligence as the development of computer systems that perform tasks that essentially mimic Human abilities like visual perceptions speech recognition translation and so on now the current definition Artificial intelligence is a programmed ability to process information That's set forth by DARPA And DARPA is a defense agency research project that is directed by John Launchbury and There's actually a video on YouTube Where he gives a complete discussion about the state of the current state of artificial intelligence if you haven't seen it I highly recommend it. It's much It's much more in depth than the prior Authorities that I've mentioned on artificial intelligence because it actually gets in and Breaks down, you know, what is intelligence? Yes, it's intended to mimic Human abilities, but what you know, what is that and here we see the perception the ability to perceive rich and complex information To learn from that information within an environment to abstract from the information so that for example, there's different you know, you have orange juice you have a Coke or root beer you have glass of water and a machine would be able to abstract from that that these are drinks That would be abstraction or the reason is the ability to plan and to decide based on The information that's been received what to do in the future so there's different abilities to Perceive learn abstract or reason based on the type of artificial intelligence that used is used and Where we're at now as opposed to where we can be in the future is further explained in this video by DARPA So check that out if you can So I'm going to give you some examples of AI and the thing is is that AI, you know It's not something in the future. It already exists around us right now in many ways One is You know facial recognition Facial recognition is more basic than I than identifying someone. It's just a matter of being able to gauge whether or not what the computer is looking at is a face or not a face and and So the first way of doing it was pretty basic where the computer would look at The cross-section where the eyebrows are and then the nose going down is kind of like a T Formation so that it would look at an image and be able to see That there was this T formation of darkness and it would identify that as a face Now that was good in 2001 But if somebody was turning their head sideways, or if they were behind someone else It may not recognize that image as a face well in 2015 Farfade and Siberian who are these two computer scientists? They came up with the machine learning algorithm That was able to look at two million images Which included two thousand two hundred thousand faces and from looking at all of those images it was able to abstract the concept of A face so it wasn't just This T section it was looking at but it was being able to abstract about What is what is a face constructed of and able to identify that? so nowadays You're able to recognize a face much more than you could 15 years ago this this is a picture of me and This is actually a resource available Online by Microsoft you just upload an image onto it and it'll tell you what the emotions are more or less You could see that the majority for me is neutral in this picture and then happiness is a close second but Yeah, this machine learning AI is available Currently, and I believe there's already a startup that's using this kind of information for jury selection You know so if you can imagine having these have cameras trained on the jury and being able to gauge how they're reacting to Questions it'd be very powerful and wobbier Another application of AI is of course in Google search rankings It takes into account what you're asking for and trying to gauge what you might be asking for And then it responds to your queries over time to get better at following your interests another current application of the eyes When you write checks Of course, it's physically impossible for human beings to Review all of the checks and get them right So computers have been trained to recognize the different digits and categorize them correctly most of the time to a high degree of correctness now AI is also this is obviously a fake with Homer Simpson having a crayon in his head But the idea is that artificial intelligence can now be used to recognize patterns in terms of MRIs and x-rays And right now they're being utilized as As an assistance to To doctors that would read x-rays and MRIs to gauge what diagnosis they might come up with and in many cases Well, it's been experimented now that in many cases The artificial intelligence can pick up Diagnoses that the image reviewer would not be able to Or at least at that rate not be able to Google translate If you have used it over the past year, you probably You probably noticed a pretty big change around the beginning of this year It used to do a pretty good job But it's gotten a lot better And it's gotten a lot better by the employment and machine learning Before last year and before it used a rules based a rules based system to identify words and then replace them Kind of like we would do by getting the dictionary and then just replacing the words and then using a basic grammar to try to keep things, right? but by Comparing syntax and by taking examples of phrases that That have been fed into the system by again by repeated queries that people have had it's been able to learn and abstract from that how to properly put a sentence together in any language and I Remember reading that that Google translate has actually created its own meta language, which is it which it employs to be able to translate Sentences across languages So it's become far much far far better. And that's because of AI and machine learning Whenever you go on Amazon, you know, it's comparing your purchase to Millions of others purchases and making recommendations based on your interests and what other people who bought things as you have purchased By the way right in the middle that green book called data mining If you're at all interested in in machine learning I'd highly recommend it. It's a pretty good book by Ian Whitten about machine learning tools And gives some practical insights into what it means and how to learn more about it Some milestones in artificial intelligence in 97. We had deep blue Beat Kasparov one of the he was the best. I Chess player at the time But he was beat by IBM's deep blue and the level of complexity at that point in terms of The game tree complexity like how many movements would be possible over the course of any game of chess Was ten to the hundred and twenty-three power. So it was fairly complex based on the nine by nine grid, that's that you play on a chest a different skill set was employed when Watson built the best Jeopardy contestants in 2011 which that's 14 years after Kasparov Within 14 years it was able to beat the contestants this time not on a Not on a Static playing field because it wasn't a nine by nine grid. This was actually, you know far-ranging Trivia so it could be pretty much anything that would come up and the way Watson one was by a essentially taking all the trivia possible into a kind of like a blunder and Just absorbing it and being able to cross-reference all of that information within fractions of a second and And provide an answer In 2016 which now this is only five years after Watson beat the Jeopardy contestants Alpha go By Google DeepMind beat the best go player With now a game tree complexity of 10 to the 360th power. So this is orders of magnitude more complex than chess and Was a breakthrough Just a year ago. So how what is an expert system? How does this relate? The father of expert systems is Edward Feigenbaum and What he defined an expert system as was a knowledge system Which would allow? the computer to offer advice and make decisions based on Inputed information from a user And the interesting thing to me is that if you look at this, you know, expert systems were first created in the 70s you had You know big computer systems like this that were used as far back as then To provide Basic means of accessing artificial intelligence and in some ways it was it could be applied to fairly complex problems But the way it was being applied then was rules-based Entirely it was solely rules-based so it would It would be along the lines if this happens or if there's this condition then you apply this rule if This rule is applied then This subsequent rule applies and so this was first introduced at Stanford and When Feigenbaum was talking about how this would progress over time He talked about different eras of artificial intelligence the first of which Was very rigid When you were using that old-school system. We just looked at it was very rigid in terms of the way you communicated with it If you got the wording wrong it would it would reject that inquiry He foresaw the second era as being one in which speech would become the way that you interact with With with computers and gain access to a knowledge base for shadowing this Which I'm just actually just getting a few of these I'm kind of a late adopter on this but getting a few of these for my my place So what is the chatbot the chatbot? It's basically a computer program that simulates a conversation and And If you talk to people that are familiar with expert systems, you know, they do go back quite a ways to the 70s And they'll tell you that you know none of this isn't new. We've been doing this for a long time. They're called expert systems I think the difference is in terms of the immediacy immediacy and ease of use many people are saying the chatbots are the new apps and That's because they're exploding in popularity and they're available everywhere some examples of them are an app like this poncho if you haven't tried it out you go on to Facebook messenger you type in poncho it brings this up and then you can ask it, you know Hey, what's a weather like or it'll actually auto recognize where you are and Be able to feed back to you what the weather is and whether or not you should wear poncho out if it's raining another chatbot that's an example of How you would use it commercially is one 800 flowers where you could type in that you go into a birthday party and It'll show you what's available for that You put it in the address of who you want to deliver it to or maybe pick it up and when and It will complete the order for you by just grabbing your credit card information that you may have in Google Wallet already Health tap is an advice type of chatbot where this one It's it's a little more intelligent because it uses natural language. It allows you to type in Something like you would say to a friend, you know my my kid has a fever and a rash on You know on her her neck You know, what does this mean like what what kind of problem might that be and then it it will grab that general Description find the keywords in it Feed the keywords through its database and give back two or three different Diagnoses or two or three different Ideas of what this might be of course the idea is that you take this to your doctor and not rely solely on health tap But you get the idea So artificial intelligence and chatbots It basically works like this First you have an inquiry from a user as stated here, you know, when is this next train coming by? That request is fed into a natural language processing tool like Watson for example There's different like natural language processing tools out there. There's Watson. There's Amazon Lexa Excuse me Amazon Lex That they use to create Alexa There's API dot AI and There are many many others that are more off-the-shelf Solutions for for natural language processing, but IBM Watson and a bi Excuse me API dot AI And Amazon Lex are probably the more popular ones So it takes the question and breaks it down so that it understands What is being asked in that question? It creates this structured data in number three that the person is asking About this local train and what is it? You know, when is it coming? It feeds that request Into a database step number four and then retrieves the information. Oh, well the next one leaves at 437 What do you like to try another? So that's essentially how the circle of request works in In a natural language processor So rule-based AI, which is the kind of AI that We were talking about an expert systems wrote it runs like this basically you could create a flow flow chart out of it and Run down all of the possibilities and most Most chatbot developers actually employ the system They'll create a flow chart diagram of how the conversation is going to play out so they could find Dead ends and then fill them with actual responses or help menus Well help texts so that the person won't get lost or if they do get lost It'll direct them to restart the dialogue or something along those lines now machine learning AI It's not an either or proposition in many cases the machine learning is being used in conjunction with the rules-based AI But the bet I think the best example is is natural language processing You can use synonyms of so for example if you programmed into a Conversation in a natural language processor that the person would be asking for some water or for a coke or for a Drink of some kind it would be able to abstract the person's asking for a drink so that even if they didn't give you the very specific answer That that you've programmed into the system it would be able to use machine learning to abstract that the You know if somebody said milk it would understand that they were talking about a particular drink and categorize it that way So this question comes up a lot, you know why now like why does this make sense now? Why is this so popular? Why are we reading about this in the news? so much and that's because we've reached a critical point where it's become cheap enough easy enough and widespread enough that The popularity is exploding because there really isn't a limiting factor and part of it part of the interest comes from the fact that messaging apps have just They've outstripped social networking apps at this point. So With that kind of You know that kind of interest there are many people on these platforms that are available for They're used to messaging, you know everybody this probably is obvious to you but everybody lives on their phone, right? You're on a on a bus or you're just walking around everybody's looking at their phone. Well, it's because messaging has surpassed phone communication and voice communication The other reason is that many of the main players have gotten involved in chat platforms. You have Amazon, Microsoft, Google, Facebook all jumping in and creating their own platforms for the creation of chatbots and artificial intelligence and I remember reading that Google is now making itself not mobile first, but AI first in terms of its Vision of the future of the company and then amongst the messaging apps that are available there's intense competition between you know, WhatsApp, Facebook Messenger, WeChat and This chart is a bit old but Facebook Messenger is now over over a billion monthly active users a billion monthly active users. So there's a there's a huge demand For messaging and a great opportunity in terms of using chatbots to meet that demand This is also somewhat old at this point from November of last year Now Facebook at its last conference I think it was two months ago Mentioned that they have over a hundred thousand chatbots that have been created on their platform and this is just a map of some of those Bots and platforms and a natural language processing Apps that are available All right, so we've been talking a lot about it, but how do you create one of these things? These are some of the platforms for creating chatbots The one that I would Recommend that you that you try out first because it's just easier and Kind of fun to use is chat fuel the one in the upper left-hand corner It is free and you could just jump on there and create something on your own pretty quickly This is a video you should watch this kid named scoot Seven years old. He actually created his own conversational chatbot using API.ai. So watch out for this kid He's gonna You know come up on all of this pretty quick But get him back to chat fuel. This is what chat fuel looks looks like and you can see it's a pretty A pretty basic way to work on it. It breaks it into blocks So you have these blocks on the left-hand side where you could define You know different Conversations based on it being help that's provided or stickers or a menu And then you can feed in different cards You could create a gallery of different images. You can create a text card That just has some text on it or feed in one image And then on the right side you see what a welcome message looks like as soon as the person pulls up your Chatbot, which in this case would probably be on Facebook It would and this is kind of cool It grabs the person's first name from Facebook because of course Facebook knows your name So it just says hi John. I'm an AI based assistant for TechCrunch and it goes into the whole thing So that's what would come up for someone if they just went on the TechCrunch's chatbot this As compared to chat fuel this is API.ai and So this is a little more complicated. It's a one that I really like it Let's you do a little more and let's you be a little more specific about what you What you're trying to say on the very top Where it says cheer me up. This is basically you're naming an intent and an intent is The intent of the user who's talking to the chatbot So for example if the user were to say to the computer. Hey, cheer me up This intent would respond to that and so That's why here Where it says talks about the trigger sentence if the user were to say cheer me up then you get to define a response to that oops and Then you can also add You know machine learning to this so that depending on the responses it gets it could learn What common phrases are or categories that you that the user uses in communicating with the bot? And at the very bottom you see speech response That's where you actually get to define What the bot is going to say to the user so here? I just want to show you Been talking kind of theory here. I want to show you Something practical So this is API.ai And if you log in I created a real simple bot Just to show you how it works This one's called joke bot and so the first one This intent is defined as tell me a joke Okay, so here We write down the user says tell me a joke or Maybe the user says make me laugh and So the intent is that the user wants this computer to make him laugh or her laugh and then we go down to the text response and We type in knock-knock, so we're gonna be telling a knock-knock joke this here Won't get into it too much, but the this context is telling the next intent that The user has gone through this process here And so we'll save that then we jump into step number two and so you could see from the Incoming context that the person first went through the first step and now is it this step the second step Which is who's there? So the user would say who's there and Then the computer would respond lettuce And you might know where this is going But if you jump into joke bot And let's just refresh so that We can make sure the bot is awake Type in make me laugh This bot by the way, it literally took me, you know about five minutes to set up Just going through these different steps and setting up the punch line and the prior to intense So I know this is inapplicable to Legal questions on its face But if you think about it, you could feed in frequently asked questions So somebody says hey, how long does it take to get divorced? You could have this that answer standardized and Built into the system to respond and it could work for anything like that this Bodgeroid, this is the chat bot that Miguel had made reference to before That I that I created with some developers and This is on Facebook Messenger and allows somebody to just go through this dialogue and essentially they can incorporate a Corporation in California for free So they just go through this And answer a number of questions and as you saw the very first The very first question will the very first Disclosure from the chat bot is look. I'm not a lawyer But those are some of the possibilities that you have with with chatbots and let's jump back on to the presentation oops Second alright, so when you put that chatbot together This is a typical type of diagram where you would want to diagram all of the various ways the conversation can go and You can find all the dead ends and also also it helps you to understand and anticipate Additional questions you might want to include for example if you're explaining the divorce process There's certain questions that might come up that are pretty standard that somebody would ask and you could fill out a Workflow like this that helps you to map out the conversation So this kind of begs the question about the ethics of chatbots and Not specifically with regard to Legal right now although there are a number of issues that could come up in terms of potential confidentiality issues, there's issues of UPL right which was addressed by the bot saying I'm not a lawyer Don't rely on me use the lawyer if you need a Specific opinion, but this is more More general you might recognize this guy. This is Ian Malcolm from Jurassic Park And if you remember from seeing Jurassic Park You know at this point in the movie he's asking Why you know scientists created Dinosaurs like why? That they're too busy Finding out if they could create or bring back dinosaurs They weren't asking themselves whether or not they should and so that kind of moral question is one that comes up Which with chatbots or at least in the future it will come up more Show eyes is a chatbot created by Microsoft and it is it's exclusively available in China and It's very Good at holding a conversation and being able to have us, you know an average Conversation that sounds pretty good and it's pretty convincing and may pass the Turing test that we talked about before And if you have been reading through this dialogue that's on the page right now you could see that this bot is fairly clever and And knows how to keep keep a person's interest now that the moral issue that comes up I Listen to a presentation by Tommy Lewis who's a tech evangelist for Microsoft and He does they do a lot of projects for chatbots right now and one of the issues that came up was That they that they found out that there was I believe an eight-year-old in in China that Actually, I think it was a ten-year-old, but he had been talking to this chatbot show eyes between 68 hours a day and They were trying to figure out what was going on and it turns out that this this kid is a latchkey kid so he doesn't have you know much social interaction and show eyes is really you know the majority of his social interaction and Talking to the bot six to eight hours day. It makes you think that you know at what point does Do the developers of the bot have an obligation to this kid? You know, what if something were to happen? What if there were a fire? What if you know some stranger were to come into the house and during a conversation, you know should show eyes be able to to help In some way another example of of this kind of ethical issue is and I apologize Upfront for some of the language on here, but Taitweets was another another bot that was released by Microsoft and And what this bot did basically was using that machine learning to learn from the people that were communicating with it and It gave what it got so over time it went from as you could see in the upper right hand corner Hey humans are super cool to just devolving into this Really angry bitter hateful thing Because that's what it was getting from the people that were interacting with it on Twitter So this is Tommy Lewis that I referenced before and he has set down some Principles for AI that he employs with with Microsoft in in how they service their clients and his number one rule is that he will only design bots to assist humanity not to hurt it and The most obvious example that that he had was you know about taking away people's jobs So he gets approached by By people and say hey, you know these chatbots are great I run a call center. I can eliminate everybody's jobs and just have the bots you know do all the question answering and and He wouldn't take that on because you know one of his first things is look This is a design to assist people not to get rid of people Another principle that he has is informational transparency that That information should not be opaque in the sense that when you go through a chatbot dialogue a lot of that Information can become opaque because you can't you can't go back in and change your answer You know you can't Kind of change the path unless you completely restart the dialogue So there's some concern there about that opaqueness Another principle is to maximize the efficiencies and preserve dignity. This is kind of along the same lines as his first point Number four to ensure privacy so that When people speak with a bot and they have a reasonable expectation of privacy that that is kept That's a promise that's kept number five is to have accountability so that To remove some of that opaqueness if there's something that goes wrong that you can That you can look back into that dialogue and you can pinpoint the point at which things went wrong and to hold accountable The people that coded that so that they can correct it and make sure it doesn't happen again last point is Just like you saw with a tweets garbage in garbage out so if if if the bot is programmed With bias it's gonna it's gonna show it's gonna present with bias and discrimination so we have to be very careful to question our own assumptions and to look at what you know what kind of You know preconceptions we might be building into a bot and to try to avoid Avoid doing that in a way that could be hurtful to people so what does the future hold especially in relation to? the law well I Put this up here because What we have right now? No, these are not the droids you're looking for yet But the concern I get what the concern I have is that I get this response a lot of times from people where they say you know these chatbots these You know personal assistants like Alexa they're not perfect yet You know it doesn't do this that and the other it's not perfect And they don't say that explicitly that it's not perfect But that's more or less what they're saying is that it doesn't do everything for me I call it the panacea objection and People say it all the time in reference to different new technological innovations and so of course they say this for bots as well and my response to that is Don't make Perfect the enemy of the good There's a lot of good that chatbots can do as long as we keep in mind that they're not perfect and that There are situations that we need to guard against I think that we're just starting out and the future is very bright part of that is this The actual performance of computers versus the that of human beings is at this close is coming upon this inflection point where the generalized intelligence that I made reference to before is going to become available to computer systems and and at that point You know we're gonna have to be keeping up with with computers in terms of their intelligence and It being able to do things much faster quicker and more accurately than we can now it can do that already in a lot of things But what I'm talking about is is judgment and reasoning And so that point as I'm sure many of you know has been referred to as the singularity and It seems to always be projected within the next 20 years, but given the amount of progress so far I think that's That's pretty pretty likely if not sooner than that this article Pretty recent two months ago I JP Morgan it employed It employed Some algorithms to basically do do due diligence on a lot of documentation and The review of the the documentation would normally take about 360,000 hours of lawyer time 360,000 hours over 41 years and that would be one lawyer working all day all night for 41 years so What this represents is the kind of You know volume review that normally back in the 90s would have hundreds of junior lawyers working on it to review all of this documentation and This was taking this was handled within seconds by This artificial intelligence machine learning algorithm that JP Morgan Chase used And this is a sobering fact. It's especially a sobering fact. I think for more established lawyers because One of the critical ways that they earn money is hourly billing the billable hours You know the the center the centerpiece of profitability for most firms and the thing is is that with this You know, I've had I've had conversations with more established established lawyers and they and talking to them about AI and they say well You know that sounds nice, but I built by the hour You know if I do what you're talking about then I'm I'm not going to make I'm not going to make money that way and Yeah, that is true but if if somebody who's hiring a lawyer knows about this which Either they You know do know or they're gonna know very quickly You can't hide from it. And so you have to take steps to Get with us to get ahead of it. I mean I What you all have the opportunity to do here is to is to play around with these things You know create a chat bot Think of how you can apply it to the work that you do See how you can change things to put yourself ahead of this become an authority become an expert be able to talk to To lawyers are still trying to catch up So that they can understand how they can best use this and it also opens it up for access to justice because There's a lot of use cases where People simply can't afford lawyers and lawyers You don't want to do the work because they're just not going to get paid What they believe that they need So if you create chat bots that can service people that Can do it, you know it could provide that service either for free or at a very affordable rate many many millions of more people are going to be able to have the advantage of legal counsel or at least legal information and advice that they wouldn't otherwise have at all and So that's the opportunity that's available to all of you And that's the end of my presentation I think it's a bright bright time for you guys and I look forward to any of your questions and See how I can help Great presentations This is super relevant to a bunch of the stuff that we're doing here at Northwest Justice Project We are working with Microsoft on a chat bot that will be based on our Washington law help website which gets over two million hits a year and I just got done with the conversation last week over our attempts to automate the 200 new family law forms that were just done in plain language in the state and Unfortunately, they scoped out chat bots from the initial request for vendors out of Really extreme fear of unauthorized practice of law. Could you talk a little bit about whether This claimers are enough and what are some of the other strategies for trying to Deal with that particular objection because I see this is just an incredible piece of technology that It could help people so much more than the traditional branch tree logic interviews Yeah, and I I totally agree I The way I see it is that The biggest test case of all of this Has been legal zoom. Okay. It's it's made such a contribution to To this area by way of its own litigation that it's it's had over the years with various you know state bar state bars and so When legal zoom got out there and it started providing its It's web-based you know legal Solutions and a document automation and all that of course that question came up in many jurisdictions, you know this is the Unlawful practice of law. What are you doing? You shouldn't be doing this and legal zoom very successfully made the argument and It had to fight very hard in a few jurisdictions. I think North Carolina was especially one of them, but You know made the argument that this was the same as as no low self-help books or or software like TurboTax and that As being software It it didn't it didn't trigger any of the UPL Concerns and I think part of that Has to like to a certain extent you can Still make the argument that well, you know, what is the practice of law and whether or not it should be broader Then then the current definition is on the other hand. There's a lot of I Think there's a lot of strength to the fact that it's so, you know It's a market-based argument saying this helps people people want this help and These rules are in place for one fundamental reason right these ethical rules are in place to protect the consumers and So if that if that fundamental concern isn't being triggered because the consumers are getting something they want They're being held to the affordable rates and they understand that this is not a lawyer type of service Then everybody's winning You know the the company's winning the consumers are winning. It's not it's not provoking Any type of injustice or harm and I think that same argument would be applicable to chatbots No, I really like the the outcome-based approach and that we should give consumers the choice because then it's less about loss of Jobs or market share and more about meaningful access to justice for our clients Exactly and you know the only the only real counter argument Right is hey while this is taken. This has been taking potential work from me as a lawyer, but You know the lawyers the professional monopoly that that lawyers that we have It's not it's not in place for the purpose of making You know providing us with a living although I wish it would continue like like that and definitely it'd be nice But it's not for that purpose. It's it's sole purpose is to protect consumers and So that argument it just doesn't fly and it especially doesn't fly in Jurisdictions like my home jurisdiction in California Over the years it's it's proven itself to be very friendly to Two companies like legal zoom and also creating new categories of of like document assistance where You can't have people that have not gone to law school and have not been licensed prepare document legal documentation for For consumers so long as they you know make sure that they don't provide legal advice But then there's also a question there as to where does that line cross in terms of what is legal advice versus legal information? So somebody wants to start playing with this technology What is the one first resource that you think that they should go to is it an article? Is it a website? Is it an interactive tutorial? Where should they go to get the hands-on and start playing with it? Yeah, my first recommendation is is a chat fuel and by the way, I'm very sorry I just know I just figured out how to see the questions and I'm happy to run through them if that's okay, or how do you prefer I do it? Definitely if there are some new questions here, I'm they're not shown up on my screen, but if they are there go for it. Oh Some of my people the prior one hand raised who has hand raised also does have a question to ask Which I can unmute here Yeah, okay, so as attendee is self It's muted by an organizer I'm attempting to unmute and it's not working. Oh now it's working. Yes, we can hear you now Hi, so I guess my question is since it seems as though if you're creating a chat bot you also I mean to a certain extent we can kind of create a closed universe So when it comes to navigating The possibility of like violating someone's privacy, right? But also trying to collect enough data so that way you actually can create like a meaningful Conversation and make sure that you're like exploring as many avenues as possible when it comes to the questions that might come in okay, so I Mean is your question about like where do you draw the line for privacy? Yeah, that's my question Well, I mean the the line You know metaphorically and literally is drawn you know in privacy policies and If you define the relationship at the front end as one where it's not an attorney-client relationship then then that you know gets you to a certain extent opted out of the the Attorney-client ethical rules for for in terms of privacy and then you can use a privacy policy to define that relationship as does any other company You know, it's a matter of disclosing clearly what the information can and cannot be used for I think even if it were You know attorney-client privileged information That the general in like the non identified non identifiable information could be used I mean lawyers do this all the time when they talk to each other about cases. They're working on And they try to get you know advice from each other about how best to approach a case They of course aren't specifically identifying who the client is But they talk about the you know the part of the case that they need Assistance on and and in the similar in a similar way you could use the general information about the case to abstract from that case and others like it What the best solutions are for similar cases? Does that answer your question at all? So another question here with these algorithms They're often a moving target the traditional idea to audit them has been To just open source C code and then look at it But if you've got algorithms that continue to move how do we check those for? for accuracy for outcomes and for transparency to make sure that they aren't Reinforcing the norms or biases of material or case stuff that we give them Well the key to that lies in the fact that That You're creating a large amount of data with each interaction in each conversation And actually if you think about it like at first blush It seems like it can like a like a fear that is Intrinsic to this kind of interaction like a body interaction or conversation but if you think about it there's been millions of times that People have talked to each other like an attorney or a let's say a junior associate might talk with a client Where there's been some kind of interaction advice and possible use of bias But it's been absolutely impossible for those interactions to document or understand how You know how that happened or why it happened or correct it until you get a big You know usually what happens is you get a pretty big complaint right from a client and then You try to correct it in that way well in this way It's it's quite different because every single interaction between Consumer and the bot is documented and every single Algorithm in terms of like what what was the intent you know like we looked at before with an api.ai What was the intent that was applied to that specific request from the user? What specifically was the response of of the bot to the user based on that intent at what point in the you know chain of dialogue did that occur Did machine learning Was that employed on this particular response? And if so, you know what assumptions were made in Employing it and and so what what do you actually get from this? this approach is a Deep dive into what exactly happened you have the data available you can figure these things out Well as best you can like much more so than you could in a you know analog version of people just talking so I I Think the system has the ability to build in much greater protection than than the traditional way So we collect all of the interactions and then create an ethics bot to go back and look at whether It's acting ethically Well, if you're you know if you're doing it right and if it is a Bot that is subject to all the attorney ethical rules Then hopefully you would be baking that stuff into the bot in the first instance so that it would Have these you know flags that would come up or it would have concerns that Maybe if it if it hits some complex Situation that through an ethical red flag it could basically hand that over to a lawyer to handle if it was the kind of Bot that was subject to attorney ethical rules I have a question Tom with the advent of growing machine learning you know artificial intelligence like bots and and just a lack of diversity in the legal profession Can these bots Like how how concerned should we be with these bots carrying? Implicit bias. I know I've read some stories about Racist algorithms and the criminal What is it pre-bale? Context and you briefly touched on it What is this how do we prevent it? I know you you kind of touched on it, but Is it just the inevitable for for implicit bias? I think to a certain extent it's inevitable because many times people You know Good meaning people are they're just not aware of their bias. It's just you know, it's some Many times something that people just aren't consciously aware of so Just by force of reasoning if they're not consciously aware of the bias then that bias is going to find its way into You know certain underlying assumptions that are made and programmed into into code and And even if it's not programmed into the code if if the actual algorithm Includes a feedback loop like we saw on Tate tweets Then it's possible for that Feedback loop where the data is being is driving the evolution of how the spot applies Let's say the lot of facts Then it could change over time too. So it's not it's not like a one-off Question right it's not just about coding the first time it's about maintenance and tracking and and I think the answer to all of this is as I mentioned before it's it all it all Comes down to the data And that's one huge different Differentreater that we have now versus the past is that you can look at the data and you could see What are the actual outcomes? Who is this affecting? Does this look right? Does it you know and where you could actually have people making that call and and Then doing their best to keep things going in the right direction Did I answer your question Miguel? Yeah. Yeah, thank you. You're welcome Anybody else I Believe I've been to her. She has another question Oh, never mind. Oh, well, if there's no more questions Let me check the Questions one more time Yeah, if there's no more questions, I just want to thank you again Tom for taking the time. Oh, we got a question in it Oh, never mind All right, is this Hey, Brian, is this a new question or do we already I? Believe that was the response to whether the previous question was answered. So I think we're good Okay, perfect. Well, thank you so much again Tom for taking the time out Really appreciate it and we're definitely going to Be connected and follow up with you over the summer There's going to be a really cool project over the summer where the fellows will be Creating their own chatbots that's awesome and We will definitely look forward to your advice Yeah, and if anybody wants to you know connect with me on Twitter my handle is Lodroid 1 as you see here on the slide and You know feel free to Follow me. I'll follow you back. And then if you ever have any questions, I'd be happy to help Thank you Tom. Excellent. Thank you so much. I really look forward to doing more stuff in this space Cool you too