 the slides. I am. Okay, great. All right. Good afternoon, everyone. Thank you for joining us for one of our summer webinar series. Today we are going to be talking about Legal Navigator. Our summer webinar series is all about self-help websites and tools. And we're very excited to have two speakers with us today. Glenn Rodden, who is Senior Program Counsel for Technology with the Legal Services Corporation. And Nalani Fujimori-Kaina, who is the Executive Director of the Legal Aid Society of Hawaii. So thank you both so much for being here. And I'll pass it off to whoever's starting. Okay, Angela, that would be I who is starting this. And so welcome, everybody. And I hope everybody's having a good start of summer so far. Nalani and I want to talk to you about a Legal Navigator. That's a project that we've been working on for quite some time now. And I'm going to start with a short video that our Public Relations Department put together at LSC. That kind of gives you an overview. And then Nalani and I will go in some details. I hope we've got it queued up so that the audio is okay. Have you heard about Legal Navigator? Do you know what it is? Legal Navigator is an online portal that guides self-represented leaders in a civil legal problem to help the content and service providers for their state or locality. The goal of Legal Navigator is to simplify access to legal information to provide curated, high-quality information from trust resources. The concept of Legal Navigator is to be more than a statewide website. It will make it easier for people with civil legal parameters to get legal information to make a trustable understanding, make a plan for a past forward, and share this plan with a reliable service provider. This is not a civil legal system. You can aggregate your state's trustworthy, verified, and reliable legal assistance resources all in one place. Each state can build on partnerships with other legal aid providers, state courts, social services, private lawyers, and bar associations to contribute and manage content and resources. Here will be the Legal Navigator highlighted in two states, Alaska and Hawaii. Each state oversaw its content, including topics, resources, and guided assistance. Each state concentrated on the subjects that were most important to them for the client, and built guided assistance in those specific two to three focus areas. User testing has been done in both states, and the results are accurate. Users were excited about the service and the ability to have a curated experience. They trust the information on the site. They thought the site was very simple and the guided assistance was easy to understand and easy to use. And users were happy that they had plenty of meetings to share their content. If you are looking to build a state-wide portal, this could be a way to jumpstart your efforts. The code is open source and free. U.P. is legal navigator to expand the availability of trustworthy legal information in your locality, curate more tailored legal assistance and conversion and acting plan, and make referrals to service providers to provide for their own assistance. And the major technology are essential for the legal community to narrow the justice gap. For more information and to help you get started, email legalnavadmin at legalnav.org. Produced in partnership with the Legal Service Corporation's Board of Directors, Hawaii Justice Foundation, Alaska Court System, Microsoft, Pew Charitable Trusts, and Pro Bono or Daniel. That gave you kind of a brief overview of the concept of legal navigator. Just to start out with, I want everybody to understand legal navigator is not designed to replace somebody's website. This is not supposed to replace the statewide website, the court's website, the bar's website, none of that. It was actually designed because there are so many different websites. And that's been one of the issues with all of those different websites. Where do people go for trusted information? You saw the photographs in the video from the ideation workshops that we did in Alaska and Hawaii. And one of the things that in both places, it was very important to people, was that they have trusted information. If you just go out and you do a Google search and you ask for something like, what do I do for an eviction or what do I do for divorce? You're going to get all kinds of results. You're going to start out with the paid results at the top for the people that bought the ad words. They're going to be from all over the United States. It's just not going to be useful for people to do it that way. So the idea was we wanted to create a site for a jurisdiction or a state that people could go to and know that all of the information there has been vetted, that all the providers there are people that have been vetted by the access to justice community in that locality. And so that was the idea behind Legal Navigator. Now, so many of the websites that we used to have, you started out looking for information just by doing a search term. And that's kind of like what you do with Google and it returns so much information and in no particular order that it's not all that useful. So we wanted to do something different with Legal Navigator. For one thing, we did include topic-based navigation because there are a whole bunch of people that just want to drill down from the very beginning. They've got an idea what their legal problem is. They know what they're looking for. And so they want to be able to just go in under the topics and drill down and find that information. So Legal Navigator is designed to allow you to do that. You can set it up just like the normal website where people can actually look in information by topics. But we also wanted to do more than just allow people to find things with topics. One of the things that Rebecca Sandefur's research for the ABA years ago did was to show us that a lot of people may have a legal problem and they don't even know it's a legal problem. They just know they have a problem. So the idea behind Legal Navigator's guided assistant is to allow people to go in and describe in their own plain words what their problem is. And then using an artificial intelligence engine developed by Suffolk Law School called SPOT, we are able to help narrow down what they're actually looking for and possible legal problems associated with what they describe. And when they do that, then we want to provide them with a guided assistant to help them get to the exactly the information that they need. Now, the way that we got started with SPOT was that we got some information from Reddit on one of their legal forms where people could come in and ask for answers to legal questions. And anybody could go in and answer the legal questions, but there were a couple lawyers that were moderators. And we got in touch with them and they gave us something like, I don't know, a couple of hundred thousand inquires that people had made so that we could start educating the artificial intelligence or the machine learning engine. Now, the way these engines work is by looking for patterns. I mean, we've all read about artificial intelligence and looking for patterns, but they need data. The more data that they get, the better that engine can do. So one of the things that we also did to start educating SPOT on legal terminology was to work with Stanford University and they set up a project called Learned Hands, a very clever play on words that law students and attorneys could sign up and then they could look at some of the questions that people had asked on Reddit and they could classify what those actual legal problems were to help it get smarter and smarter. And the idea is the longer that we use this, the more people that use this, the smarter AI can get. Now, SPOT is available to any nonprofit that wants to use it. Other websites are already using it like the Massachusetts Resource Finder uses SPOT to return that. And it's based on LIST, which was the next generation of the National Subject Matter Index that we've been using to index legal content on our website for quite some time. So this is something that we've all been working together on. And I'm hoping that maybe you get an idea of how you might be able to use that or how a legal navigator might work in your jurisdiction. Now, after they go in and they choose a guided assistant, because there may be three or four that are returned by SPOT, once they pick the one that's applicable to them, then what we do is we ask them by size questions to be sure that it's right. You can see on the left side here where it talks about, you know, it's a guided assistant interview about evictions. Is this what you were looking for? Do you want to continue? And you can see at the bottom, it gave a little snippet about what an eviction is. So you can actually use this interview to begin the education process for whatever their legal topic is. This was built using A2J Author, and then it's served up inside a legal navigator. We've been working now so that you can also build these interviews with DACA symbol or with legal server. And I think those two projects are just about done so that you won't be limited to A2J Author if you wanted to build these. These use branching logic. This is not part of the AI. I don't want anybody to think that, oh, all you have to do is connect with Spot and then it's going to do the interviews for you. There's still a lot of work that you have to do to get these interviews right. But the idea is you ask them these by size questions like, are you the landlord? Are you the tenant? And then just take them through just like you would as an attorney or a paralegal interviewing somebody about their legal problem to help you find what kind of information they need to resolve their problem. That's the way that the guided assistant will work. And when you finish with the guided assistant, we want to give you an action plan. These are going to be the different steps that you need to resolve your particular legal issue. This is an action plan that was returned on the system for divorce separation and annulment. And you can see there's a little plus sign beside each one of those. These expand so that there are multiple things in each one. But we like giving kind of a snapshot to people so that they don't feel completely overwhelmed. They can see, oh, we've got five steps here that I need to do to take care of this. Now, if we divide up one of those like step three, you can see that there's a lot more information in there. And you can put link to resources to videos to things they should read to court forms, whatever you need to do, you can put those in these action items within each step. And then you can see up at the right hand corner that can actually check them off as complete. They don't go away, but those show is complete. And so one of the things that you saw in the video is we'd like the idea that you can take this action plan and share it with a legal aid provider, an attorney that's doing unbundled legal services, or anybody that you actually want to get help, a pro bono attorney that can help you with this problem. And the idea would be if you had six steps there and you've completed four of them on your own, but you need to help with two, then it'd be very easy for the person that you're sharing your action plan with to see what you already have learned about this, what you've already done so that they're not wasting your time and their time repeating things that you've already done. And you can either download these or you can print these so that you are able to have this action plan to share with somebody or to refer back to. Now at this time, we don't use user accounts, this is identifying information, as they go through that guidance system, all it's doing is getting to the stage to give them the action plan. We're not creating user accounts, we're not storing that information. In the future, that might be a functionality that we would decide would be good to add to the system to do something like that. But right now, the decision was made to make it where people can be comfortable that what they're doing here is private. And that's also why we want to be able to download it or to print it so that they won't lose everything that they've done when they've gone through there. All those links, everything like that will be in that action plan. Now, it can have lots of other resources. As I said, this is an aggregator of the resources. As you'll see here, you can have featured resources. These are the ones that you think are the most important. You can have a link to forms. You can have links to the guided assistants, to the organizations that are going to help them, or to self-help resources at the court system. This is a way that people don't get overwhelmed, but they can click on the tab and they can drill down in the resources this way so that they see any different type of resource all listed there together. But they're not overwhelmed with one big, long list that they have to scroll through. And by having the featured resources, you can figure out that if somebody's got this legal problem, this is what they're most likely to be looking for. And you can put that into the featured resources. We've done the system using WordPress, which is an open source content management system, so that there's lots of people out there that can support the system. Lots of people know how to use the system. And there's a management console inside of WordPress, where you do all this with the content. So you designate what the featured resource is. You designate that when they're looking in this particular area, these are the organizations that are surfaced. And Nailani can give you a lot more details about that when she talks about it because I'm on this from the design and implementation standpoint. Nailani is on it more from the we actually use it and here's what we wanted to do and how it works type of standpoint. Nailani, I'll let you talk about the recent developments and some of the other features. Thank you, Glenn. Glenn shared a bunch of the things that are already on this chart. The biggest thing that's changed from the start of this development of this legal navigator is that it was rebuilt on WordPress. So the back end is a heck of a lot easier to use than what it was originally built on. So it is definitely usable for other states and those that may have less technology background in terms of being able to in order to update the website. It is a simple login system where you can go in and it's pretty user friendly. It's user friendly enough, as I always say, that I as someone who has not been trained in any technology can go in and make updates. The second thing is that, in the partnership with Alaska court system, what's been great is that having a legal aid partner and then a court system, we have different needs. And so within those needs, there's been lots of discussions about things that we want to be able to turn on and off, but still keep the sense of what the site looks like and keep some kind of commonality. And so there are definitely increased built-in options for display and customization in terms of certain information that, you know, that a state may or may not want on their site. So, you know, wanting to share that was one of the things that came out of this partnership and working really closely with Alaska and Junie Sacco from the Alaska court system is actually also on this call. The next thing is, as Glenn talked a little bit about is that we have worked to do integrations with legal server case management system, guided navigation, and simple justice are also in progress. So there are some opportunities there to potentially have content living in a legal service, you know, legal services program system that then can directly connect via API to the to legal nav. And that's something that is in the works right now. We also are looking at a soft launch in the next couple of months. We are getting closer. We're waiting just for a couple of more APIs to get cleared with legal server and hopefully we will be there in terms of the soft launch. And yes, it has taken a while for those of you that are following this project and lots of lessons learned over the years. The next things that are coming up for us are really the dock assemble integration for living guide data assist that we're hoping to start in the next few months. And then we also have once the site is a little bit more live and gone through soft launch and some initial testing, we are also going to be working to get what we're calling alternative language switches, which is basically the opportunity to switch the site to another language. If you have languages that you have a lot more materials on and translated materials on it, the site does use Google translate. We know that there are different opinions about whether Google translate should be utilized. We do think that Google translate at least can be a starting point. And part of what we've tried to do is we are trying to put in language there ahead of time so that people know that they should really seek the assistance of a legal service provider or contact somebody directly if they speak a language other than English. So again, those are some of the things that are coming up within the within legal navigator and we'll be happy to talk in just a little bit if you have questions about the program. Next slide please. So in terms of implementing legal navigator in your state, Glenn talked a little bit, talked about that it's not intended to be a resource provider, but kind of getting resource providers together. You know, resources have to be already available in your state. It is really kind of a way for people to get to those resources. There are lots of materials from the National Center on State Court around this around this, but also note that it is it's stated and it's pre you know it's pre setting up all of these things. So I would say it's a great place to start, but definitely reach out to different programs or listen to all the various webinars that are happening this summer on self-help resources because I think there's also lots of lessons to be learned from all of the other, all of the other portals out there right now. Governance is going to be important, especially if you have multiple players providing content in right currently with both Alaska and Hawaii. We really are the ones that are providing the content. So Legal Aid Society of Hawaii is providing that in Hawaii and the court system is doing it in Alaska really as kind of the initial stages, but as you start to look and figure out what you want to do in the state, knowing who's going to be in that governance about what content is going to look like, who can be on the platform will be part of those, should be part of those conversations. And then finally, you know, understanding as part of that who gets on board, right? Who gets to participate on the site, how that actually looks. And again, those are going to be conversations that will be important depending on your state size and depending on how this is looking for your particular state. Glenn, did you want to add anything on this? I did, Nailani. A couple of things that you covered that I really want to emphasize people. And the first one is the governance body. You really need to decide who's going to be in your state. You know, if you've got a really good active access to Justice Commission, that's a really good place to start because we don't want this to be just a Legal Aid project or just a court project or just a bar project. We want all the players working together. That's the whole vision behind Legal Navigator. It's like we said, to aggregate all that. And then the reason we point you to the National Center for State Courts materials is the Public Welfare Foundation several years ago did what they called the Justice for All project and they gave grants to several states so that they could actually work on planning around providing 100% access to people with civil legal needs. And one of the things that was done for this was a really good inventory guide to help you look at what your existing resources are because it's just not right to put up a website like this, take people through all the process of the guided assistant. And then when they get to the end, you don't have any resources to support what they were actually looking for. And so this is something that before you start with something like Legal Navigator, you should really do this inventory process to be sure that you've got resources in the areas that you really want to help people with. Like I said, if they need this particular form, you want to make sure that form exists. If they're going to need an explanation in a video, be sure that's all there. And find out where everything is. And so to me, that's one of the most important things in making sure you've got all those resources. And like I said, there's really good materials already done for that Justice for All project for the National Center. One of the most important things in starting a project like this as well, in terms of the larger conversation that Glenn just talked about in terms of governance and having the right players being involved, you still need to figure out internal supports for whoever is going to be in charge of this project. There still needs to be people that are going to be content creators and content managers because while we may have content out there, the content may not be in the format that's going to be workable within this particular platform. So it is really important that that is there. I say it this way, technically inclined staff, because I don't think you necessarily need computer science majors or those that have their technology degrees. But you need them to assist in creating the guided assistance through either A2J, legal server, or DACA assemble, which will be ultimately the goal by hopefully the end of this year, three ways in which you can build guided assistance. You're also going to be technically inclined staff to update the site. While I can do a few things here and there, I think as you want to grow the site or find ways to integrate with other programs, you're going to need somebody who at least has a sense to be able to do the updates on the site on a regular basis. And then finally, which is really critical for all of these programs is that you need to figure out the resources to support the staff and the platform maintenance costs. With a switchover to WordPress, the platform maintenance cost, we really believe is not that high, where the resources are really going to be spent is supporting the staff in order to maintain the site and ensure that you have ongoing correct content and managing the partnerships that will continue in this project. Next slide. Oh, go ahead. Well, I was just going to say, I mean, our next slide is the question, but one other thing that I wanted to mention is that one other thing that has come to my attention that you really want when you start this project is a sustainability plan. That is something that even if you find a Microsoft or an LSE or somebody to give you startup money to get the project up and running, it is something from what Nailani was telling you all those resources that you're going to need internally, somebody has to provide those. And it really is not a good strategy to give them to existing staff that already have full-time jobs. So this is something that I always encourage people from the very beginning of the project to come up with a sustainability plan. Who's going to be contributing? What? Who's going to pay for the server costs? Who's going to be paying for the person that actually manages the content? You know, any other hosting fees, whatever, you want to be sure that somebody has committed to do those so that you don't get a really good project up and running. And then it dies. We saw that in one state, New Mexico, where TIG funded this project. We had three or four legal services providers working on it together. And then after the project, there was no sustainability there. And so the project just went away. And that's not what anybody wants to see in this type of an instance. So that's just something that I would point out to you too is very important to think about from the very start. Don't get to the end of the project and get it implemented and then start worrying about sustainability. So, Ange, that's our part of the presentation. We're hoping to get discussions, questions. Melania, I'm not sure when is this going to be available on GitHub, the code for people? We're hoping in the next month, we're waiting on the legal server for one of the APIs. So the consultants are waiting for that. They want to make sure that everything's uploaded at the same time. Yeah. So it'll be on GitHub so that you could take this and you could download it, put it on whatever servers you want to run on and get it up and running. And so we're hoping that people will take the platform and use it. That's one of the things that we've always planned on doing because there's so much cost and work that went into the development of this. Microsoft put in a couple of million dollars and a lot of that was in the user testing and Pew Charitable Trust also invested money in this to do user testing for us and also we feel like it's a good platform and we're hoping that it gets adopted by more jurisdictions or states to actually use. And that's why it'll be on GitHub so there won't be any cost to you for the software. And there's also a user management guide for the content management. So that you'll have instructions on how to do that as well so that you're not just starting with a piece of software with no instructions. Great. Well, thank you both so much for this presentation. We I think everyone is muted just for purposes of having a good presentation. But if you've got questions or comments, we would love for you to unmute and ask them or raise your hand or put something in the chat. Any of those options are good or great. I'll get us started by asking, with the action plan, how detailed do you get in that action plan? And I ask, as Michigan Legal Help, we sometimes struggle with what level of detail to get into because with action plans, you get into like jurisdiction specific nuances, shall we say. And so I was just wondering how you're handling that. I guess this is probably for Nalani, but if Glenn has ideas as well, it would be great to hear them. It's a great question, Angela, because I think Alaska and Hawaii do it differently. And I think that's one of the neat things about the platform is that we've kept it really at a much like just a few simple questions so that it kind of gets people to the right forms or at least gets people started. So we will ask things like do you have children so that we provide in a divorce to provide information about what happens with custody and things like that. On the Alaska side, and I really do invite Jeannie to jump in since she is on the call, they've been very specific about how to fill out particular court forms and the information that you need. So I think the slide that you saw where there was detailed information about what you needed, what information you needed from your spouse, that's one of the things that Alaska's been doing with it. I think the question really goes to, again, the point that there's so much flexibility within the within the platform for states to decide how they want to utilize the guided assist. And again, I think there's just there can be different many different approaches. And I think it's one of the things that we're curious about in terms of, you know, what what feedback is going to be on on the different approaches for what what people are seeing. And it was some of the things that were initially tested and people were excited about even the different approaches, right? I think, again, it just depends on what your state and what your state resources are. And that's going to be the question that your state is going to have to answer for itself as to what they want to use it for. Let me let me just add one other thing on that. And and Angie is that the way that I think it really also depends on your conceptualization of what the site is for, right? We at least I believe in how we've approached the site is that it will not be able to replace assistance for for the most complicated cases, right? For those that have more barriers. And so we really we wanted to be really at a much, you know, kind of like, you know, simple place to get people to the right resources as a starting place and not necessarily so detailed on like figuring out that jurisdiction question, because there's so much nuance in that, particularly when you when you raise the point about the jurisdiction question. Again, I don't and I don't think Alaska goes into that that far either. I think it's just really about how do you fill out the forms and what's the information that you need. So I think we always want to keep that in mind. And I think we try to keep that in mind, at least with the design of of of what the guided assist walks people through it's really trying to get people to the right forms that they need the right information or at least thinking about the right legal questions that they need to answer before they even get started. I don't know, Jeannie, are you able to unmute and share? Sorry, I don't want to talk over you. No, no, no problem. No, no. I think it's a really interesting question. And our administrative director and I actually just spoke with the New Mexico Access to Justice team about our self help services. And I think for our family law interviews, we were trying to be more detailed and pretty. We have a group of phone facilitators who answer family law questions, and we tried to somewhat duplicate what a phone call would be like with some more detailed questions and then more detailed answers in the family law realm. And I think part of that is we have a lot of family law resources and for a more sophisticated user, they can probably go to our website and find them. And then maybe the next level user hopefully could go through the interview and get some pretty detailed answers in their action plan, kind of through the steps they need to go to. But that would be someone who is comfortable with reading and, you know, having all the steps, but we would still have the in person option for people who maybe learn better through listening or having a discussion. I think for some of it would be each agency that would want to use it could pick what level they want to do. But we did try to see it as a artificial intelligence discussion or interview with the hopes that it would be like at the end would be similar to what you would be told if you were talking to a person. That's kind of our ultimate goal. Although in some subject matter areas, there's maybe just not as much detail as there would be in family law and it will be, they will be not quite as detailed. And the concern that I think we haven't that anyone would have that Glenn mentioned is you then need staff to update all that. And so that's also a decision you have to make based on resources and how much staff you have to maintain it post building the interviews. And I do think Nailani mentioned, and she's so right, you do need tech savvy staff. And I think you also need folks, there's some really good lawyers who maybe aren't or people with legal background who aren't maybe logic and logic trees and that sort of thinking isn't their strength. And that's a pretty important skill for someone who's going to be building the interviews, someone who enjoys kind of logic puzzles. So, but I think, yeah, Nailani, I think you did a great job describing the difference between the two jurisdictions. And I think we also looked out because Alaska and Hawaii, we've been meeting once a week for about four years. And we're, get along very well. So we luck out with great team members. Great. Thank you both. How about others on the call? Are you thinking about trying to implement this in your state? Do you have questions? Rochelle, I see you unmuted. Yeah, I'm interested in knowing what the interplay is between your portal and your legal help site, especially if your law help site has a lot of content already and maybe it has fillable forms and it has, and then the portal also has, it seems like the portal also has like a built out directory. So where would, do people get confused going back and forth between the two sites or when is there content like content that would live on one side and not the other side or how do you avoid the duplication? Yeah, so the way that the platform actually works is that it actually connects to whatever the other platform is. So right now, any resources that we have on law help will connect into the portal. So it's the same, so it's the website address for whatever document is on law help. So you could do that kind of interplay. So you could have, if you use the legal navigator as the portal, you could still have law help as in the background to host all your materials. This would just be a different interface that you could then also maybe connect to the court's website or to other community websites that you have. So I think as Glenn talked about it, right, it's the idea is that it can be kind of an aggregator or, you know, it's the entry point for legal information for your state that might connect all of these different platforms. With that said, in our state, our plan is actually to move, is to actually go get off of law help. And we're going to be utilizing simple justice through the legal server to connect a lot of our resources up as a very different way of approaching. And so we will likely be off the entire law help platform and just have this platform as being kind of the connecting place for a lot of our resources. So I think it's going to depend on what states want to decide to do. This is a choice that we made just because I didn't want to have to update materials in multiple places. And again, what the thing that we're working on to get into GitHub is really the API information so that once we update content in simple justice, which is again a legal server platform, it'll automatically update the API on the on legal navigator. So those are the kinds of things that at least we've been trying to think through. So I think it just really it's going to depend on the state. And Alaska is actually doing a different approach that I think would be another option for states or people who are interested in the platform. We have a very robust website already. So on those featured when Glenn showed a screen that had the featured resources tab and actually the screen he showed the first featured resource was Alaska court system information on. I can't remember his debt or housing. So we're using that as a featured resource. So if someone goes to the resources page, we're directing them back to our website. So like Nailani, that's our solution to only have to maintain one source. And we're putting very few resources on that page and just getting people to our web page, which we're maintaining. And then we're focusing on the interview building in Alaska to be something an ad to our web page is through the interview building that guided assistant. And when you click on an external resource, Rochelle, it opens up a new tab. It doesn't do it in that same window. So then people can close out that tab and still be where they were in their action plan. We didn't want people having to move back and hit the back button to get back to where they were top of things. So other questions. I guess if no one has any, I'll ask another. Do you have any learnings from the spot integration that you would like to share? I think, you know, that's for many organizations that are looking at doing that or it's new and so any guidance? I'll let Glenn talk about the technical side of it. We've had things happen. I just tested it this morning and it was actually doing a much better job. So I think, as we've always said, right, as it learns, it gets better and better and better. So I think it is an opportunity, but there is still a need, at least on our side, to put in keywords. It's still critical for the search patterns and everything else. I think Spot is learning and there's opportunities. One of the things that we saw that we know is really true is that the way that Spot works requires more than just two words. And we're so trained with our Google searching to just put in divorce versus like my husband just told me that he wants to kick me out. And so I think that's one of the things that it's going to be interesting to see how people use it because if they don't type in more of a story, then we have seen some issues with Spot, which is why we have the keywords. There's also an added search feature, but I'll let Glenn talk a little bit about the technical side because I think he was working more closely on that. Well, and the other thing, and as you probably know, we're not where we want to be so that we're sending back any results to Spot, which is where we really need to get to get it educated. In other words, we send them to Spot. Spot looks at the engine and sends them back to us. But when somebody clicks on something, we need to be able to send that back to Spot to say, ah, you know, when you send back these three results with this question, nine times out of 10, they click this one, Spot gets smarter and knows that should be at the top of the list type of thing. We're not really there for that. We have designed it as, you know, Nelani said though, it will also look at your keywords that you've indexed with each resource in WordPress so that you don't have to depend solely on Spot. You know, if you've got something really unique that Spot's not returning, but you put in that keyword in your WordPress, then it will also look at that and find that resource as well. So it's not a one or the other. You can actually do both of those keywords and that. And right now, too, if I remember right, the Lewis only goes down to not Lewis, look, list only goes down to the second level. And so that needs to get more bus so that people can drill down further into different legal categories as such. So, but, you know, it's an evolution. You know, that's why they call it machine learning. The more that this does, more of their work and David Caleruso and his group have been very responsive when, you know, issues have arisen or something like that. Okay, thanks. I had another question. As part of this project, you guys did all this great user testing. And I know that you've incorporated the results of the user testing into your into what you designed. But I'm wondering, is there any opportunity, like, are any of those reports just available to see or to share out with the community? I feel like so many states are doing user testing, but we don't necessarily see what what what they find. It would be so helpful for the road as we're all upgrading and making changes to see what other tests have happened. I don't recall whether we've ever published any of those. Rochelle, I'm, you know, that was something that, you know, was done by Nation 72, initially, and then Pew did some. Pew is getting ready to release some of their evaluation materials that they did. They did a presentation too on this of Equal Justice Conference, where they looked at not just legal navigator, but several of the other portals and legal navigator. So, they, Lonnie, do you remember, have we ever published any of the Nation 72 work? It was very specific. Right. And I think that's the thing, right? I think the reports were so specific as to, like, did this, did the placement of this work here, right? I mean, which I, which I can, which I can hear from you actually could be useful, right, for other programs. I'm going to get something for us to think a little bit more about. And I'm going to, if I'm looking at Angie going, maybe it's something to work with Alison on in terms of, like, I mean, you know, aggregating, right, not just for our portal, but for other sites, right? What are the things that work and don't work so that, you know, so that programs across the country aren't struggling with the same, same design, you know, design, quote unquote, I'm not saying flaws, but I mean, but issues like that users come up with. The same design questions because we all have, I bet we do have very similar questions in the end, you know, like, do buttons work best at the top or the bottom? You know, how many buttons is too many? Is our pop-ups good or bad? Or when are they good or bad? So, that's a great question, Rachelle, and a great idea, Nalani, to, to share more of that. Well, and a lot of what it surfaced to was accessibility problems that we needed to correct, you know, getting, I forget what the terminology is now, I'd never heard of it before, but we didn't have it. And so it was something that you needed in there. And then going in, if you've got any pictures and, you know, doing the tagging those right and all that. So that helped us a lot with accessibility, too. Yeah, I mean, to that, I mean, one of the interesting findings which I think we had never thought about was that when you have a banner button on the top and the bottom, right, it repeats for the person that has accessibility, right, so that it actually is more problematic to have the same information on the page, right, even though, for quote unquote, prettiness or user design, right, we think it looks kind of nice to be able to have people accessing the information both at the bottom of the page on the top of the page. So we removed the banner that allowed for access on the bottom of the page. So I mean, there were small things like that that I think, you know, as you ask the question, I mean, they're great questions because not sure maybe that is standard, you know, non-designed for everybody else on this call, but it was a lesson that that we learned, right, with that accessibility testing. I think some of the information too, maybe I don't know if there's a report or not, the ideation workshops that they did where people identified what was so important to them, it was amazing to me how similar they were, both in Hawaii and Alaska. And one of the things that we tried to do with this is look at three different user groups, not just the end user that's trying to do their own divorce, but also the people that are the legal navigators in their community, they're the trusted advisors. And, you know, for example, in Alaska, one of the people that came to the workshop was from the post office there, because the only government building anywhere close to that community was the post office. So anybody who had a problem came in and asked the postmaster there how to solve that problem. So one of the things we wanted to do with legal navigator is to be sure that the community navigators that are helping these people would be able to use the system as well to help get these people to the right information that they needed. And then we want to make it good for the providers because, you know, they're the other user group. They're the ones that want to surface their information. So we tried to be attuned to that. And so those were really those were two day workshops in each state. And they're really interesting to me. They overlap in what people wanted with the trusted information. They want to be sure that their identity was safe. They wanted, you know, the resources that would really help them. You know, it was an interesting experience. It also showed me that, you know, this is what Margaret Hagin has been saying forever. Before you start building the software, bring your users in and see what they want. Don't guess what they want and then bring them in to look at it. All right. Any other any other questions or comments? You've got our contact information. If anything occurs to you later, or you just didn't want to say it in front of the group, you can reach out to us. So. Thank you. Well, thank you for the opportunity. Thank you. Thank you so much, Nalani and Glenn for coming together today to talk about legal navigator. Just as a reminder, we do have, let's see, three more websites left in our summer self-help series. They are every Thursday at 3 p.m. for the next three weeks. Next week, we have statewide self-help websites 2.0. We're going to hear from a couple of programs who are doing rebuilds and relaunches of websites that have been around for 10 or more years. Then next up is a user-centered design and self-help websites webinar. We'll be hearing from Graphic Advocacy Project talking about the fundamentals of the UX design process. And then finally, the last webinar is called Not Another Form, Innovative Self-Help. So other tools besides document assembly that are being used creatively to help provide legal self-help assistance. So I hope you will join us for those webinars. As always, they will be recorded and available on lsntap.org. And I'll see you next Thursday. Thanks.