 All right, I think we have everybody in from the waiting room currently. We're excited to have mark lordson here to provide us a discussion on interactive checklists. I am going to show you what mute. We still have people coming in. You're on mute. If you want if you care to be. But I will get started. Good afternoon everybody. This is mark lordson with capstone practice systems. Our topic today is interactive checklists and I'm indebted to Shelly for suggesting this topic because I've written about it recklessly in a book and some articles as a good idea. But I've never really delved into it in much depth let alone built a few that are that deserve that title. So I hope you will join me on this journey through some interesting ideas. Today's session is going to be pretty rapid fire we have an hour. It's going to be somewhat conceptual, but I'm going to be showing concrete examples and going under the hood a bit to get into the details of how this stuff works. And I should say that this these sessions in general are aimed at a nonprofit legal services audience. I noticed that there are others that are from other sectors of the community we are most welcome. Everybody supporting legal aid somehow you should do so either technologically or financially, but we encourage you to to join our community. I would I would think most people would agree that checklists are extremely handy, especially as you get older and life shopping lists are are pretty inescapable. So in grocery shopping or other, but in areas of life like aeronautics leaf preflight checks are critical. Making sure that all of the various systems are working before you leave the ground and endanger your passengers. And in surgery, it's been shown that even very simple checklists can have remarkable impacts on the incidence of failures and problems. So wash your hands. Do the obvious things make sure you confirm that you're operating on the left leg not the right leg, or the right patient. Count the sponges before you're you so somebody back up. Make sure you haven't left something inadvertently inside of them that's going to cause trouble. So checklists are all over the place and they certainly have a role in law. They're not simply things to do, they're not just to do list necessarily. There are also things to think about things to consider while you're making a choice or making a plan things to take into account. And we talk a lot about knowledge management these days. In any organization checklists can be used to capture and to share complex processes so that you don't have to run around looking for how to do something, or track down somebody who knows better than you. Very important in various contexts, surprisingly effective, but very often overlooked. There's a good article on Wikipedia that includes a list of various airplane crashes that have been attributed to simple checklist failures somebody just forgot to check something. And they took off with that checking and caused a lot of deaths in the process. By a tool go on the money you're familiar with the surgeon at Brigham and women's hospital in Boston, the checklist manifesto that documents some of those kinds of failures and also the surgical impacts that I mentioned a minute ago. A lot of related concepts to do list as I mentioned is a, what we usually think of instructions more generally are forms of checklist to guide somebody through a process even how to videos on on YouTube to deal with a plumbing problem or other common issues. Reminders recipes of course have lists of ingredients and steps to follow. Sometimes with variations built in if you're a vegan you know you can substitute this for that. But generally are, are checklists of sorts their processes that have been formalized and procedures that have been organized into a computer computable form. We also think of flow charts and various kinds of workflows or process maps as checkups as checklists, and then check ups is another variety where you're really stepping somebody through a number of considerations to be taking into account. And in facing a situation they may be unfamiliar with. On the right here I've got a image of one way in which we sometimes conceive of checklists as organizing jobs into steps and then the steps into one or more, one or more tasks. I think of the checklist kind of fall on a spectrum. There are most checklists we deal with our static and they don't change as we use them. They're not customized to us individually, it may be on paper. They may be online, but they're just there there are a list of things that you follow, and then you're done. But as you move up to various forms of automation. First of all, you hit the idea of personalized or customized checklist, where there's some interaction with the user that results in the checklist being customized to their circumstances to their goals. A little more automated is the kind of interactive checklists which are the focus of today's talk, where there's some dynamism in them, whereas they're being used in real time, the checklists react as you check off things other things appear. Some things disappear because you've already done them. And if you move further along the spectrum you might get to what we could call intelligent checklist using some of the breaking tools like chat GPT and all the large language models where we're so infatuated with today, where it's not simply a preordained workflow of what to do that's interactive but where it's more intelligent where it's going out and noticing things about the world and organizing knowledge for you. And we're not going to get into that today but I think that is inevitably in our future. There are several other kinds of application types that are quite similar that have overlapping features, obviously various forms of form generators or document assemblers are quite common in the legal world. We've used these dynamic questionnaires that take somebody through a complex landscape of issues and decisions to be made, perhaps resulting in a document or two or three or 20 being assembled various kinds of analytical advisors that do the same thing not for the sake of generating a document, but giving you some insight into how to understand a problem or decompose a situation so that you can act appropriately. There are also various kinds of decision support tools that use similar techniques to organize the thinking through of options and factors and considerations and evaluative perspectives that may be necessary to appropriately make a good, a good choice. Users in the law context certainly clients who perhaps in combination with their attorney would be guided through a process of describing their circumstances and deciding what to do and actually doing some things on their own behalf. One representative litigants of course is a common target of the kinds of applications we talk about an MSN TAP and in the legal aid world. Then there are the advocates themselves the practitioners the client servers in programs whose own processes often present tasks to be done which the particular practitioner may not be expert in. The checklist there is helpful, and even for experts as we've seen even for expert surgeons, simple checklist that state the obvious very often make the difference between a happy result and an unhappy one managers of organizations and other other staff have all kinds of things they need to do that are amenable to checklist formalization and people who are handling calls, whether it's technical support staff or hotline managers, having a interactive checklist that can be dynamic for a given call given conversation, very often can result in more complete advice and assistance for somebody. In the use cases in law. On the right here I've got a snap of the article I wrote 20 years ago about this idea of smart pads on the on the web that were back in the day when it was possible for the first time to be online without being connected to anything and thinking about how that could change law practice how could how would how could change trial work where you can cross examine a witness with a interactive transcript or checklist in front of you as they answer questions, it can suggest other avenues for further investigation and interrogation. And for things like opening and closing a case administrative processes preparing to oppose somebody or to negotiate on behalf of your client. All these things there's routine steps tests that one want to do is a best practice to do that kind of a job. The checklist is one way to organize your activities so you're doing the right thing. And then there's things like getting a travel expense reimbursement which some people find very, very painful managers often need to onboard new hires or off board people who are leaving. There's interesting complex processes involved in choosing procuring technology and services, and then deploying the results. There's a lot of other things that I've been a little bit against, which I'll mostly talk about today. There are, there are, there are tasks to be done, like filing and serving a complaint or responding to a complaint that seem obvious to anybody who has been active in the legal process for a while, but for the average person are far from obvious and possible pitfalls and problems. For instance, what, what should I do when I go in for my divorce hearing. What should I say well how should I address what should I not do, what should I not say. Now checklists have some common ingredients. And these are, this is a list up. The list is just the variables pieces of data that are different from case to case obviously if a checklist is to be interactive. It needs to elicit from the user some information about their situation their, their goals, what they've already done what they have yet to do, what have events have occurred that might impact the recommended next steps. Then there are computations or inferences you can draw from this raw data that you might gather from a user. Then there are conditions, if then branches in the logic so if you're in this county make sure you do this when you go to the courthouse but if you're in this county. You have to bring a different form or speak a different language, use a different vocabulary to accomplish the same result. Then the third major ingredient is repetition. It's very often the case that there are certain things that need to be done more than once are certain forms of information that need to be gathered more than once. Maybe there are multiple children involved in a case or multiple defenses in an eviction or multiple guarantors in a in a financial transaction. So those are the key ingredients variables, computations conditions and repeats or repetition. And with them you can accomplish quite a lot in any of these contexts I've described so far. Then there are many variations. So a checklist interactive checklist can be presented via a web app, which I'll show some examples of, or a smartphone app, or it could be a chat bot intelligent chat bot, or a system that interacts with you via text messages, or a smart speaker. Sometimes that is simply text, but often they can present infographics or provide some forms of interactive visualization as part of the process of working with the user. It may be standalone just to do what it does or it might be built as part of a broader application, maybe as part of a document automation process. You generate all these documents. And now you've got some tests to take care of to actually proceed with using them in the in the case. And finally they sometimes are done as simply one time a single one and done. You use it and you're done with it you throw it away. Other times it's a process that goes on across time, you want to be able to interact with a checklist use it for a while, save your, your responses save the state of the checklist, and then come back and resume. So I've done this. This has now happened. Now what should I do. One way to display this kind of knowledge to a user is the flow charts or decision trees. These can get quite complex as you can tell from these examples. One is a divorce process flow chart from Michigan legal help. And for somebody who's comfortable with this mode of expression. You can follow through and see where you go in the various arrows and where branches which branch you're on and get to an end result and understand the landscape of the of the overall process. This one from Arizona court court help is a little more graphical and simplified. But once again it's it gives you a lay of the land gives you an overall view. There are plenty of apps online and elsewhere that you can look at that use this basic idea this one is from the gavel marketplace gavel used to be called document as some of you know it's one of the document automation tools. But this is a a check up for insurance claims in Canada. So if you're about to have to deal with an insurance company. And give you some pre advice about how to best do it and what to expect and how to deal with the company but it's not acting fairly. This is yet another variety this is my own side project called choice boxer was kind of a checkbox for for choices, where you have options in this case you know choices of an expert system platform. factors or considerations that differentiate them where you they are different degrees of goodness from your perspective on these different factors. And then you may also have different degrees of importance how important to you is functional completeness versus ease of use for instance. And moreover this can vary by person so if you have a team of people trying to choose a product, you can use this to organize the collective thinking kind of a checklist of considerations take into account. I'm going to show some more specific examples now to get away from my slides. Let me let me open this first one this is an application I helped build about eight years ago for a self help center in San Diego. That was helping people who couldn't afford lawyers or didn't want a lawyer to file for a conservatorship on behalf of an adult child or other relative who was not able to manage their own affairs. This illustrates some of the basic ideas. I think if I click on it, I should go right to it. Now this by the way is. Just minimize some of my zoom chrome here. This is the first introduction for some of you to this law help interactive service I think most people know about this. This is a wonderful national service been around almost 20 years operated by pro bono net in New York, and it hosts thousands of these kind of modules mostly for document automation. A lot of guided interviews that lead to a document, a lot of other kinds of interactive tools to help people deal with legal problems either on behalf of clients as an advocate, or as an unrepresented litigant yourself. What I just clicked on is for generating all the paperwork needed to produce a limit of conservatorship petition in San Diego, and I'm coming in. I think I have to log in since my session died. Let me do that quickly. You can use, you can use this service anonymously, or you can establish an account. And screw up your password. And the advantage of having an account is you can save your answers. So I happen to have some time in the past. A set of answers for a case in connection with this particular application. Let me the Lynn family, totally imaginary. And I can bring that up rather than having to type in front of you and make all kinds of mistakes. So here we are. This is the application itself. It's a guided interview. This happens to be using a tool called hot docs. It's really this one is is for advocates navigators more so than the unrepresented litigants so it's in a self help assisted prosaic setting. And it's designed and this is not been actively used in a couple years I don't think there may be another replacement for it. It's really used when someone comes through and says hey we really need to get a conservatorship for Mary or whoever we're doing it for help me generate all the forms and undertake the process to accomplish this goal. So the advocate would sit down with the client or the unrepresented litigant and go through these questions so it's kind of a checklist it's a it's a interactive questionnaire. So in that case you're pursuing where are you, what course will you be in what courthouse I'm sorry, who are the parties the conservatee and the conservator or conservators who is going to be the petitioner. I won't dwell too much on this but this is a common application type. We see quite a bit and as I mentioned there are thousands of these on will help interactive or LHI, as we call it, and it just drills down upon all the information about the various parties. There are circumstances and what's going on someone's just a developmentally disabled. What medications to the use who are the relatives and friends and neighbors within several degrees of cons, consanguinity that need to be known about specified in the forums, and if alive, identified and notified about the dependency of this petition. So you will go through this entire process. When you get near the end, you'll see a list of all the various forms you could generate many of these are pre checked for you based upon your answers to prior questions. And they're organized into several rounds or tranches of filings you may need to come back to do a second filing. So this is a case where you'd save your answers resume and come back later and produce further documents, and there's a little bit of guidance about what you do next. Let's just go ahead and say okay we've spent enough time in here. We've answered all the questions we care to answer. Let me generate. There's some questions maybe that were not required that I left them unanswered. Let me go now and generate the forms, and there are several dozen forms I learned for conservatorships in California. In an average case you may need to do a dozen more. In this situation, voila, within three seconds. The application has produced a 41 page set of all the forms for pursuing a conservatorship, the petition, and all the other documents that are necessary some that need to be filed and serve some of which are under seal, etc. It's quite an impressive job to produce this beautiful set of forms for you, the saving a lot of time you answer questions once, etc. But of course the natural question is great this is wonderful I have all my forms. What do I do next. And that's where that's where a checklist might come in. So let me go back to my slides. Here I am. Okay, so that was one example that didn't quite it was kind of a checklist deprived example didn't really take us to this place we want to be, but it illustrated some of the common features the idea of variables or fields, the idea of probability, for instance in that interview if you said, I want to, I want to request a fee waiver it'll present questions necessary to elicit the information to produce a fee waiver motion and associated documents, and then generate that for you. Let me bring up another example this is a yet another complex form this one is in Washington State. One of my favorite examples. So this one is file for divorce. I'm logged in. And I'm going to once again go and say let's use some answers I have a set of mostly gibberish answers I believe for this imaginary case. But this is a similar, a similar experience right we've got a long list of dialogues or pages that somebody's just starting would need to go through. There's various kinds of instructions and guidance along the way. There are all the variables all the all the questions when will you marry, where were you married, who are you. Have you been separated. More information about this person. This game given a gibberish name. The spouse. Are you pregnant, or is your spouse pregnant. Are there any children. Are they joint or separate. Is there a parenting plan in this context you expect to need to file up file up file one. Again, once again, it goes through it has repetition various multiple forms of property various debts someone may need to deal with. And you would spend as much time here as you need to be guided through the steps of describing. The desired divorce process for the purposes of generating documents. And when you get to the very end of this one. Let me see here. Yeah, there is some guidance you know you're you're you know congratulated for doing a good job and completing the interview. You're told which documents are going to be generated and again this varies based upon your facts and your choices earlier. There are some guidance about about downloading them them, but there's not much other guidance here. Now the difference about this one example is that if I finish this. And I ask the start the site to generate the documents for me. In this case it's going to generate. Not a single PDF document with all the individual forms concatenated inside of it. It's going to generate seven or eight word documents individually as separate files that you can download and entered if necessary locally and and file, but they include instructions that's the critical step that I think is different here. So, now I've got my, my various documents to download I won't go into all of them. Let me just download these instructions. So, here we go. So this is a personalized set of instructions customized for this specific person based upon the answers they gave in their interview on this day. Right. And it tells them, here are the documents make sure you got them all. There may be more documents under a different fact pattern, and then each of those documents in turn has got a cover sheet with instructions particular to that form. So it's really a great, I get best practice for how you, how you organize the provision of instructions and guidance to a user who's not familiar with the terminology and the processes they're about to have to embark upon. So I'm going to start explaining with nice ideographs tells you what you should do next. Once again this handy room here so with a pen, if you print this out you can you can check off these things as you do them. And, and I find it just an extraordinarily useful, useful tool, lots of information all the information you need. What to do if you have more questions. Yeah. So, once again this is, this is personalized customized it's not interactive at this stage right so you're in a document is a static document, but it's taking into account most of the variations, peculiar to your, your circumstances. All right, let me get back to my, my slides. All right, one, one, one critical thing I forgot to show you, let me, in addition to all these documents. And by the way, I should give credit to Lori Garber who may be on this call. Lori is the main author of this application she's an attorney at the Northwest Justice Project, and she's worked tirelessly to build this and many other similar applications for One of the cool features she and I, and others worked on is this idea of next steps. So, on top of everything else we've seen we've got this intelligent interview that we've been through with all kinds of guidance along the way. We have the documents we can generate with with customized instructions both both in general and specific to particular documents. And then we had this new feature called next steps, and this is basically saying it reminds me I should download my documents and I should save my answers. Yeah, I'm just going to do that. I didn't change anything I don't think but I'm going to save just be compliant. So now I've tucked away the latest set of answers to the questions I was dealing with. But now when I click the next step box. The system as a whole is offering me next things I can do right here on what help interactive. So you'll notice, you may remember that in the interview we said we wanted to get a fee waiver. We believe we needed to make a parodying plan. And elsewhere in there there was a decision about whether to generate a child support worksheet and order, proposed order. So these things are now set up the system knows based upon the facts and your answer set that these steps are appropriate for you to do next. And if I now want to say okay, yeah I do want to get a fee waiver. It was not generated as part of the initial divorce process, but I'm taken to a next module on the law help interactive service specific to waving a filing thing. And what's cool is that all the information I had entered in the earlier interview, my great unusual name, etc, has been carried forward. So it knows everything about me that I've provided to the system. And it's, it's a form of a checklist of process guidance that I think is quite creative and useful. Right, so that's the, that's the file for divorce Washington forms online. It illustrates this idea of in interview customization of some instructions and in document instructions that are customized not only to the user situation, but to the specific documents that are part of the package that's been assembled for the user. One other example, this, this shows a different interface this is the a to j interface. Again it's running on the law help interactive service. I'm not going to bother opening an answer set on this one. And for those of you that are familiar with a to j, a to j is a more of a graphical interface. It's got an avatar of an advisor optionally you can have an avatar for yourself. And this, and this is organized kind of them as a pathway to a courthouse or to another form of ultimate relief. This was done as part of an ABA project I believe I was not part of it. But here you're, you're being stepped through a very, very simplified user experience. Not as much text that is not as heavy a screen as the ones we've seen so far. And I can, I can go through and enter information and say I have a problem with the rental unit or I'm looking for one. And I won't go through it all my being that no I'm not being evicted has the landlord done any of these things no. I have bed bugs no. So it's it's a it's a form of a checkup again it's stepping you through the return my secure deposit. I won't go through more but as as I answer questions I'll move along these steps to the end result and based upon my answers, I'll be guided to some useful resources to help me deal with my my situation. So think of that as an example of a checkup. And in this case again it doesn't produce a document per se it generates a link to a to a external page. So you can think of as kind of a pathfinder example. At the end of this session I'm going to show you another just very brief proof of concept I cooked up for this for your occasion that's more of an interactive checklist that actually does things as you use it. So stay tuned for that. But first let me just identify some obvious problems and challenges complications. Some of these processes can be quite complex as we know. It may be that you should do X or Y, but if you've already done Z, you shouldn't do X or Y you should do Q. So how do you code, how do you, how do you express and organize that knowledge to help somebody work their way through a process. This are are disordered order is not important. When you go to the grocery store it's not important which order you buy things, although it might be more efficient to do it in the order in which you encounter them in the aisles, but many, many processes of course have sequences built into them. And you have to do one thing in able in order to be able to do another thing, and you can't do certain things until other things have happened or you've done things. So that needs to be taken into account and built in to any kind of an application. Often you have multiple users multiple people involved multiple actors. There may be approvals necessary or some forms of delegation. And so you can also see the lawyer client scenario where there's some things the client can it should do there's some things the lawyer can or should do. And ideally those, those are organized and choreographed into a common framework, so that as as the process proceeds, each party knows to know what to do and does their appropriate steps. And then of course, there's always the question, whether these kinds of applications are improper legal advice, given outside of a lawyer client relationship. That's being, you know, many even the simple form filler applications have been accused of that, let alone the current round of of chat GTP and related tools. My own, my own philosophy has long been that we are and should be free to code the law in various ways. And at least in the United States, the First Amendment protects code and software applications. Just as much as it protects books and pamphlets and videos in other forms of expressing knowledge and sharing information. But be aware that may come up if you're doing these kinds of applications someone may may raise some questions about the legitimacy of helping somebody work through a process when they're not paying a lawyer or engaging a lawyer to do that. There's artificial intelligence. There's, there's quite a bit of useful literature I just recently read this top article about law maps, which is fascinating it's an attempt to automate some of the process of extracting the structure of statutes and other kind and processes legal processes and visualizing them on behalf of litigants. I think that's worth a look there's a link in these slides for all the stuff I'm showing by the way, and the slides I will be be available from Shelly LSN tap when we're done. And then recently I've been reading about process mining, which is somewhat new to me but so fascinating discipline, having to do with techniques to, to take to extract intelligence from large collections of event logs. You're in a situation where users or staff or other people have been interacting with a tool or doing tasks, and you've got a large collection of data very often you can extract implicit structures and charts and frameworks from them. So check that out if it's of interest to you. Pause for a moment and just talk about the tools to make these kinds of applications. We've seen the end user experience. Both of an A to J author application and a hot docs one. I want to give you a little look at how those work under the covers. There are plenty of other tools popular now or tools like brighter gavel with a key toe in the auto logic spent around a long time. And as some of you may know there there are hundreds of these out there in the world. My friend Catherine Bamford had published a list of over 250 document automation tools alone. I'll be skip over to a to J. So we've seen this end user experience for an A to J application. And I'm going to go to where this thing gets where these things get built and maintained I didn't build that one but let me show you one is just to give you an example. So we built an application for handling the appeals of denials of FEMA benefits. And in a to J author which is a freely available website you can sign up for and create these yourselves. It's got all the basic features of document automation and these interactive checklist ideas. So there's various kinds of variables a universe of data points that you identify that need to be elicited from a user and used perhaps in generating a document. You've got various steps you can organize your, your questions into the specific pages themselves can be organized into specific pages that present the questions, and there's logic among the questions so based upon the answers the user gives to various questions, other questions may or may not appear. And you can visualize that flow with a built in mapper that that sort of builds a dynamic decision tree or flow chart for you of how your questions work and this is useful for debugging and designing. Once you've got everything organized pretty well you can write in the authoring environment, you can preview the user experience. And this is the example of the FEMA appeals request in this case it would I won't do it. I won't go through it but it'll, it'll generate all the documents you need to request your file and and has a cover sheet of instructions, just like we're showing before. I'll just show you the same thing behind the scenes for the divorce application I showed you. So this is the Washington State divorce application, not necessarily the latest version of it I've just downloaded at some time in the past. So this consists in a similar way of a large collection a much, much larger collection of variables. These are all the text variables there are a number of variables there are date variables there are true false variables. These are all the moving parts that are used across the whole range of forms and questions and various computations. These are all the answers that may be drawn from those from the answers that may be given by the user. And, but I want to show you the instructions document. So, and very quickly, this is a typical template in a tool like hot docs. It's got variables, you know for today and other things. So if you're in King County or if you're in Pierce County different abbreviations will be used to describe the court form because they have different names in those different counties. So that's a simple example. But then where I told you that there are on that list of documents there were seven I think in the example I showed. But if in fact you've got a spouse that needs to be served outside of Washington. There's a form, and there's another step to be done, but that only gets included in your instructions if it is applicable to you. And Lori has done a great job and building in all these variations to produce an incredibly personalized, helpful set of instructions for you. One thing I think is especially interesting is she's got a section in here to handle the possibility the user has left out answers to some important questions. They just make answers required on these applications. But if you do that, the user will not be allowed to proceed. If it's a required question has to be answered. And there might be some information a user needs to go look up or find or doesn't want to answer today, but they want to proceed they want to save their answers they want to generate a draft documents. And so this section goes through and quite a bit of detail touching on various pieces of information that might have been necessary. For a complete set of forms to alert the user in the instructions, you know, you really got to go back and answer a few more questions, regenerate all this stuff, and you'll be better off, and you're more likely to achieve the results you want. So just an example of how you can use conditionality and various forms of variables, even repetition inside of a document template to generate this experience of customization. Let's see okay. There are, as you might expect there are commercial checklist builders I haven't really looked at these very much. I'm going to go and create nice checklist like documents I'm not sure they offer any of the interactivity that I was showing here today, but check them out. And of course there are tools like SharePoint, which have built in facilities for doing quite sophisticated workflows. And if you have if you're a SharePoint organization you may be familiar with them, or you can you can you can implement your, your checklist ideas using SharePoint. So I want to wrap up with a closing example and then we'll have some time for questions or comments if there are any. This goes back to the conservatorship example. And I remember being quite confused myself as a non practitioner in that area about how it all worked what do you do with all these forms how do you get them filed. You can imagine the average litigant who's been helped by somebody at a self help center would be somewhat at a loss to know how to handle this. So they're all quite good online resources from the court and other providers that in plain language explain how to do these things how to answer common question what steps you do what do I do how do I fill with this form. And that's an easy example of a static checklist. It's it's one size fits all. It just tells people, this is what you do in general we're not talking about you we're talking about the general person who's facing these kind of problems. There can be guidance inside of the instructions but that's about it. So this is where I was thinking. We had used that that that module I showed earlier of a conservatorship petition form preparer that generated these 41 pages of forms so you've, you've done that, and you're at the now what stage. So, this is the, I just spent a few hours building this but this is an example of what I would consider an interactive checklist. So, what do I have to do now. I'll fill out the forms what's what do I do next. And this is just a screenshot of it but I'll bring it up live. Once again this is running on the law help interactive service. It was built in hot docs. And in a moment I'll show you how it was built. But now this is a, this is a one page interview. So you notice there's no, there's no outline here because there's only one of these guys. And it's one level it's quite simple. So, what do I do now okay I need to make two copies. Okay I've done that. Having done that. These other things now become visible you notice before I checked that everything else is great out and non actionable. So I'll say yes make two copies good. Now, I see this thing bring the copies to the probate business office. Ah, what's this little light bulb over here. Now it's a bit of guidance. So this is a bit of help contextualized to this task. And it's got some text editor telling us what's going to happen, but it also has a link. And it shows me where they, the probate business office is and when they're open, etc. Let me go back. It's always fine with. You can't see my toolbar but that gets in the way sometimes. Okay, so let's keep going. So we're going to say all right we follow those instructions. Okay we're going to bring the copies. Now we need to pay the filing fee or apply for and get a fee waiver. Let's say we pay the filing fee. Notice that the apply for and get a fee waiver has disappeared if I had chosen another path. It'll apply for and I succeeded in getting it. I don't need to worry about paying. So a very simple illustration of this conditionality among tasks to be done. And again, this is highly, highly simplified just to show some ideas. Now having done that got my fee waiver I brought the copies. I now need to actually make sure I get them back stamped by the court with a docket number presumably in a hearing date. So I've now done that. And you could provide a little bit of encouragement to the user saying great you've done it. Good, good going, keep going. All right, now I need to do two things. I need to serve a copy on the proposed conservative tea. And I need to at least mail copies to each of the identified relatives that are living and other people that have to be notified. So these are two steps I have to do both of them. And having done those now I'm eligible to take the next step which prepare and file the certificate of service again, a real example of this would would take you down the path of maybe a sub checklist of how do you do that. And what is entailed and what's the process of creating a certificate of service and maybe more than one. But you can also insert some textual guidance to the user along the way. And then finally, they need to attend the hearing that's been scheduled. And once they've chosen that task, the system can prompt them to describe how it went at their hearing. So this is mainly an interactive screen. You don't really need to go anywhere. There's not necessarily a document to generate, although it does have a built in document assembly feature. So that any stage, depending upon where you are in the process. You can go and generate your checklist as of that moment printed out. If you need to take it with you offline and see what remains to be done downstream. Let me get back to one more closing page screen. So to recap. I think we can agree checklists are really good. Nice things to have like to see more of them. If they can be customized to the to someone circumstances and their goals. That's that's even better. Right. It's, it's even more useful. You can interact with you in real time and change and advise you and guide you and remind you even better. They can be somewhat tricky to build. Let me show you, I promised how this, how this one was built. So I need to go back into into hot docs. Forgive me for knowing what I'm doing here. Okay, we were looking at the divorce one. I want to show you the checklist one. So this is going to be examples. Okay, this is what this is what's running on the server. This is my local authoring environment. And not much to show, but I think it's worth showing. Once again, it's got a collection of variables. Mostly these true false variables which correspond to actions, the user may take and various kinds of formatting structures. And then there's an interview which is basically the overarching thing that runs. All it does is ask one dialogue this so called welcome dialogue. And this guy right here is where all the action happens. This is the dialogue that presents this experience presents this interactivity. And the way it does it is by just having these various actions true false variables present. And then inside of it it's got a script. It's got a script. It's going to hide a great things, initially. And then as people answer questions as things are checked cast in a cascading way other things are unchecked. So this is, it's showing you a little bit of a support this yay in bold, you saw, and if you know what you're what you're doing with a tool like hot dogs or gavel or any other kind of a tool a to J. It's not that hard to create these reasonably interactive experiences. And you can imagine going quite quite a distance but I wanted to get that out in front of you, before we wrapped. Let me stop there. Shelly you've been monitoring the chat if there's any questions or comments. I would certainly welcome them. And I'm only yours. There haven't been any in the, in the, in the chat I think everybody was just so stunned by all the information coming through. So we do welcome questions now this is your chance to get your questions answered. Jump in. Go ahead, I was just going to ask. So, many of the examples are I guess all of the examples have been focused on client services. Yeah, do you know of any, any checklist being used for attorney on the attorney side so to help them meet all the paperwork for a case for example. I would assume they're out there but I haven't, I'm not aware of them, and I didn't uncover them in my brief efforts to organize this talk. So if anybody in the audience knows of some or wants to identify them that'd be helpful. But as you can see I think the basic techniques are applicable, whether you're helping a client or an unrepresented litigant deal with the problem or you yourself or your colleagues are doing that. The beauty of many of these tools are that they are no code tools, you know, it's, and, and when I was in law school for example I did attend some training for several of them so they are relatively easy to learn, as opposed to having to learn to code, but it's a way to get guidance for more clients and we can help in person. Exactly. And sometimes you there is a little coding involved that hot dogs example I showed you you had to get down and actually write some, some script, but it's reasonably straightforward once you once you've been exposed to it, and platforms like and brighter are, are much less Cody. So, I'm curious if anyone has related ideas have to have encountered these kinds of applications elsewhere or thought about trying to create them. I see Dennis is in the chat saying would we consider using GPT tools to do a first draft I think absolutely I mean that's all happened in the last few months. I'm sure people have already experimented with saying well tell me what I should do in this situation and GPT would come back with a beautifully formatted set of steps here the 12 things you should do. And then you might have a conversation you might say well I've done the first three, but I'm stuck on number four how do I do that. I think that's inevitable and, and those who talk about no code those really involve almost no, no application building whatsoever at the risk of potential hallucination, or, or misinformation that these things can sometimes throw. We do have a question from Catherine in the chat have you done any checklist to be used with case management systems such as legal server. I have not but I think that's a good good connection certainly integrations with tools like legal server. I know will help interactive integrates with legal server so you can have a case session in legal server, and then press a button and take all the data from that record and transmitted so it pre populates an online interview. And I think case managers in general naturally have have these kinds of process management tools but I have not personally done that. I think going the other way as well I would think from, you know, a website and have the interview and then populate fields in, in the case management system. Yeah, and we're seeing that a lot you know with Clio, of course there's there's integration now with their own in house document management document assembly system. So the opportunities to cross those worlds I think are are greater than ever. We don't want to have you miss an opportunity to ask questions but if we don't have questions will let everyone get back to their day. So, speak up now or send a question to us later and we can see if we can get it answered for you. You'll have the opportunity to review this recording if anything went by too fast. I'm happy to react individually. I think again I've proselytized checklist interactive checklist for many years but I haven't had the chance to think too much about them. So I, I thank Shelly for giving me the chance and for you to put up with this, this presentation. Well, we're not getting any more questions so I just want to put a reminder out that the video will be posted to our YouTube channel tomorrow morning. If I have the materials from Mark will also get that posted as well. Eric has just thrown a question in the chat what about flow charts to help people reach conclusions. Yeah, that's a good one I think I showed a couple of examples of flow charts they're much more in the nature of processes as opposed to decisions. But we certainly see those all over the place. And so you can think of a checklist as a kind of flow chart in textual form. I'm especially interested in the in the in the use of interactive visualization, whether it's a flow chart or this idea of a choice box to capture complex interacting ideas and coming to an appropriate defensible decision. So I have put the our events page in the chat, and I want to remind everyone that that's where you'll find our upcoming events we have a glitch right now on times. So, I'm not sure it's not displaying accurately but we will make sure that the registration information has the correct times listed for you. I have just finalized details for a webinar coming up that will not be recorded so you must attend if you want to hear it and that's going to be on office ergonomics. So watch for the announcement coming out on that. And the bonus is that the expert will be doing live assessments of four offices so you can submit pictures if you want your office to be assessed and your office may be chosen. So that is a service that can cost a couple of hundred dollars so this will be an opportunity for you to take advantage of that so watch for that announcement coming shortly. Thank you so much, Mark for doing this webinar for us. Thank you everyone for attending and have a pleasant afternoon. We'll see you next time. Thanks everybody.