 Hello, everyone. Welcome to LSNTAP's final webinar in our summer self-help series, just kind of a fun, slightly less formal, kind of straight up informal webinar series on different things related to self-help. Today we are going to hear from three, from folks talking about three different projects, and we called this webinar not just another form because there's so much talk about forms when we talk about self-help. We wanted to highlight some non-form projects. So that's what we're going to hear. That's what you're going to hear about today. My name is Angela Tripp, and I'm the Director of the Michigan Legal Help Program, and I'm going to be talking about our What's Next Text program. But before that, you're going to hear from Beth Richardson, who is the Director of Resource Development at Lakeshore Legal Aid. She's going to be talking about their cool video project. And we don't have her in real life, but we have a lovely pre-recorded session from Triana Kalmanoff, who is a UX and service designer at A2J Tech. And she's going to be talking about the Minnesota Kiosk project. So you may have heard bits and pieces about these projects, but we wanted to kind of bring them all together as a fun conclusion to our Summer Self-Help series. So I'm going to hand it off to Beth. Thank you. Hey, Sanj, I appreciate it. Let me add one more thing. Sorry. We are going to do Q&A for each panelist after their section, except you can't ask Triana questions because she's just a video. So if you have questions, we will pause for them for Beth and then for me at the end. So don't be shy. Either raise your hand or unmute yourselves. We do ask that you stay muted until that point. We are recording this for the Alice and Tap website. All right. Now you can take it away, Beth. All right. Thank you, Angie M. As Angie said, I am Lakeshore Legal Aid's Director of Resource Development, and I oversee and manage Lakeshore's fundraising, grant writing, outreach, communications, and marketing for the organization. In 2021, we applied for and received a Legal Services Corporation Technology Innovation Grant Award to produce high-quality screencasts and animated instructional videos for people who are self-represented as well as for our clients. Myself and Lakeshore's grants and innovation manager are the co-leads on this project, and we are working with Briefly, which is an animation consultant and collaborator for scripting, styling, and design, storyboards, draft animation, and final animation production. So we are in the process of completing 11 animated videos and three screencasts. And for those of you not familiar with term screencasts, that's adding a screen recording directly into a PowerPoint presentation. So you would see the narrator kind of sitting at the bottom of the screen, usually the bottom right talking to the audience. We cover various legal topics and some of the topics we're covering on these animated videos and screencasts are how to get child support, changing custody, best interest factors, paternity, tax foreclosure, how to approach DHS appeals, requesting supervised parenting time, how to serve in file group of service, and so on. So we kind of put our heads together and came up with those various legal topics that we thought would be helpful to our audience. For each topic, we enlisted the help of Lakeshore attorneys and staff to generate the content. We also did something, some user testing research with former clients before we even started to create the actual videos. And so this is really the kind of the soul of the project. The goal of this research was to be informed by client feedback so that the videos resonated with the target audience, which was Michigan residents with low income and had various legal issues that they were looking to get information on. The purpose of the study was to obtain subjective responses from clients regarding the style and the content of these videos. And the study was conducted by online surveys through type form, which is a web-based platform that you can use to create surveys without needing to write any code. We sent out two different surveys at different stages of the design process. The first survey focused on style. And what we did was we used examples of previously completed videos from our collaborative briefly, and they've worked with various legal aid organizations around the country. And we used kind of snippets from those to get feedback from those participants. And then the second survey focused more on content, including the pace, characters, narration and readability. After we had gathered some information from the first survey. Some of the findings from the surveys were the audience wanted to see characters that were more realistic than cartoony, having characters that were diverse in race, ethnicity and occupation, have characters that were warm, welcoming, professional and dignified, have characters that are large and easy to see, have text that's clear and easy to read, have designs and distribution methods to be mobile friendly, include closed captions or and or text narration and videos, and include animation and motion graphics and videos because the clients felt helpful. We then took that feedback and we incorporated into creating storyboards with the content we had generated from our Lakeshore team, review the storyboards, made edits, did a final review on the content and then added the closed captions. The videos we've completed so far are posted on Lakeshore's YouTube channel and also on our website. And once we're completed with all 14 of these videos and screencasts, we'll be sharing those the videos and the outcomes with the Michigan Legal Aid community. We're going to do some more client surveys to get additional feedback on completed videos. We're going to integrate the videos into staff training and that's something we've actually already done, we have orientation for new hires, been doing presentation to them to let them know about these videos that they can use them when they're talking with their clients. And then of course, continue to advertise the videos to public and publicize it on social media and our website. So the project has been very robust and extremely exciting. And what I thought I'd do now is show you a three minute video that is completed and up on our channel and on our website. And this one is repair issues and eviction cases. So I thought I'd show you that one. And then on the other side of it, if anybody had any questions, I'd be happy to answer that. And let me know if you can see the video. We can see it. It looks good. I think we're just waiting. Repairs issues in eviction cases. Every tenant has a right to a safe home in good repair. The legal lease term for this is the landlord's duty of habitability. It's the landlord's legal duty to keep every rental unit in reasonable repair. They need to fix things like no heat or hot water or electricity, structural issues like holes in the roof or walls that are allowing rain or rodents to enter, broken doors and locks that cause security issues, plumbing problems that cause flooding or mold, sinks and toilets falling through the floor, dangerous stairs or porches, appliances that don't work, i.e. stove and refrigerator. So how do you protect this right to a livable home? One, document these issues. Take photos with a date stamp, like a smartphone, report them to the landlord immediately in writing or text, bring witnesses to look at the housing issues. Two, let your landlord know, send them a letter explaining the issue and asking for the repairs within a reasonable amount of time. Send it by certified mail or include it with your rent check. Include photos or other evidence. If they don't reply, send one follow-up. Three, contact your local code enforcement agency to make a complaint. Municipalities should provide for complaints and inspections. In Detroit, you can complain to BSEED, the buildings, safety engineering and environmental department. For other places, look up the local code enforcement agency, contact them and request an inspection. Four, withhold your rent. This can be a risk, but it's an option if the repair issues are very serious. You need to notify the landlord then put the money in a separate account called an escrow account. You can also pay for the repairs and then deduct the cost from your rent. This can trigger an eviction by your landlord, so make sure the money is available to pay the rent. Rent is rarely reduced to zero, so it's really important that you don't spend that money. If you end up being evicted for not paying rent, the housing conditions may be a defense to that eviction. Try to talk to a lawyer if you end up in this situation. Contact Lakeshore Legal Aid and watch our video on when and how to hire a lawyer. Okay, so we've, as I said, we've been really enjoying this project and we have just a few more videos that we still need to complete, but I'm happy to answer any questions if anybody has any. Beth, usually follow up is to people who watch the video, do you track whether they get the assistance they need and whether they're able to navigate the process? Part of what we're going to do after the completion of all the videos is to do some follow up. You know, with YouTube, we can look at the data through YouTube, but we're also going to try to see and we're working with briefly because they have sort of a nuanced approach to the tracking. So yes, that's going to be like the phase after completion. Thanks. Sure. Thank you. Rochelle. Hi, yeah, that was a great video. One of the questions I had is, you said in your, the research that you did originally, people said they didn't want animated characters or cartoon characters, but then, you know, this is, is it, I don't know, and I don't know what platform you were using for. Is it GoAnimate or something similar? In terms of the, what we show them? No, I mean, so I guess it's two questions. It's on the one hand, they were saying they didn't want animated videos, but I didn't want cartoons in the end. The animation was fine. It was the way that the characters were being drawn and the cartoony ones, which you will see out there, and it's relatively popular, was something that we were surprised if a lot of people would do not respond positively to it. So in the animation, we try to create more realistic looking characters. I see. Okay. And was, is there a platform that you use to build the videos in? Like we've used something called GoAnimate. Is that what, is that what, like what? So briefly is the people that built the animation for us. Yeah. So they're the ones who do that work. We'll have some assets afterwards that we can utilize later on, but briefly is the brains behind the actual animation. Thanks. Sure. Great questions. Any others? Oh, Chris, Sarah. Hey, there. I was just curious, what did people think of all the cursive writing? This audience, they don't teach that in school anymore. I don't think. Well, we're, you know, the, we got positive responses when we showed them various types of way the text should look. But I think what's going to be really interesting is having these completed videos and having the audience sit and watch them in its completed form. That hasn't happened yet. So I'm looking forward to getting feedback about them as they are, as you see them now. But so far, we did get positive feedback in some, in our surveys on, on the different way we're presenting the language. I think readability is the biggest issue for folks. That was one issue that came up. Anything else for Beth? If you think of something later, she'll still be here at the end. Yeah. And I also want to, yeah, feel free to reach out to me at Lakeshore. It's B Richardson at LakeshoreLegalay.org and you go online, go onto our channel and look at the rest of the videos that we have posted so far. As I said, we have 11 up and we have three more posts. Can you put the YouTube link in the chat, Beth, with your email? Yeah. That'd be helpful. Okay. Thanks. All right. Well, then I will queue up the video of our next speaker. So this is, as I mentioned, this is Triana Kamanoff talking about the legal kiosk project that A2J Tech did for Legal Aid in Minnesota. Let me know if you... Hi, everyone. My name is Triana Kamanoff and I am a UX and service designer with A2J Tech. And I'm here to talk about the legal kiosk project. So thanks for having me on this webinar. It's a great honor. And I'm excited to share a little bit about this project with you all today. So what I'm going to talk about is what the legal kiosk project is. What is a legal kiosk? Why are they needed? Where are they located? And what can you do on a legal kiosk? And this project was sort of an innovative project that happened during the pandemic. It started at the very, very end of 2020 in Minnesota. And it had to happen really, really quickly because we had all kinds of funding deadlines. The whole project was funded through the CARES Act as a federal initiative for, you know, to address the COVID pandemic. And the whole thing was kind of a blur. And there was a lot that happened in a very short amount of time. So everything that you see was really iterative. We had a lot of different versions of it. We have tweaked it. We are continuing to tweak it. And so I'm excited to share it with you today, especially considering we're talking about innovation and how to sort of think on your feet and design a project that's really responsive to the needs of the people. So first off, what is a legal kiosk? So a legal kiosk is essentially just a locked down Windows desktop computer. It has a browser. It's an all in one. So you don't need any kind of like big clunky computer base. It is just everything in the monitor. There's a keyboard and a mouse attached to it. And it's mounted on a stand, which I'll talk about in just a second. But essentially, because it's a computer, you can do almost everything that you would with a normal computer. You can go on the internet, you can surf the web, you can access legal aid information, any kind of services, applications for legal assistance, virtual meetings, which was really, really key. And you can edit documents, you can open up forms and fill them out. So pretty much anything that you could normally do on a computer. There are obviously things that you can't do. We have locked some websites down, some x-rated sites, for example. You can't hook up a USB drive because I could have malware on it. So there's some limitations, but in general it's essentially just a desktop computer. And so why are these needed? So as you all know, the COVID-19 pandemic really moved the entire legal system virtual. And this had a huge impact, especially on court hearings, any kind of offices that would provide legal services closed down. And this had a huge impact across everyone, but especially people that were low income, especially people that lived in rural areas that did not have access to the same legal services that they used to have. And this exacerbated what we call the digital divide. Access to technology and internet is something that we take for granted if we you know work in technology, if we live in a city or if we work in an office. But this is something that a lot of people don't have access to and that was just made a lot, lot worse during the pandemic. And so here are some stats on the digital divide. This chart on the right here that you see is produced by the Pew Research Center in 2021. So it's quite recent and it just examines the digital divide amongst folks with differing incomes. And so you can see here I put them the bullet points for folks earning less than $30,000 a year. It's the lowest income category here. 43% don't have home broadband services. That's basically enough signal to conduct normal functions at home. So they don't have that. 41% don't have any kind of computer and a quarter don't have a smartphone. And I think that that one's particularly shocking for me even. I think we all kind of assume that people have smartphones. They can access you know whatever they need to on their phone. But that's just really not the case for a whole quarter of the people in this group here. And I mean even if you do have broadband and a smartphone, imagine trying to fill out a court form on your phone or trying to you know join a Zoom meeting for a court hearing that from your home it's all shaky. So there's a lot here that I think can be addressed. And so Little Kiosk Project really is meant to address this need to bring the services of Zoom, of a computer of a stable wireless internet connection, legal service access into the communities that need them. So rather than asking people to you know take a bus and find child care and go into the downtown area to go to court or find a legal aid office, they are in their libraries and they're in malls. They're a nonprofit organization's community locations that already exist that are already in their neighborhoods. And so they're located in about 69 out of 87 counties across the state of Minnesota and really really spread out all over. We have over 270 different kiosks. I apologize I didn't put this on the slide but quite a lot of different kiosks and there's two different types. So we have what we're calling a Model A kiosk which is in public spaces. They can be in lobbies and malls, really just something that people can walk up to not really know what they're about and see them. And then Model B's tend to be in more private spaces. So like offices, conference rooms, rooms that have a door that can be closed to be made private and that's really to facilitate joining a court hearing or meeting with an attorney so that you can have some privacy. But essentially they have the same functionality it's just a little bit about like where they're located and then the structure of how they're mounted. So you can see here this is a map of all the different locations. They definitely have some clusters in the urban areas but you can see it's really spread out. We have them even all across like the corners of the state and the green pins here are Model A's and the blue pins are Model B's. So you can see we have a good mix of both. And I should also mention that the Model B's have a printer attached to them, a printer and scanner which the model most modelists do not just because of the limited space that it requires. All right so I have that here. So for Model A you can access the state wide law help website, law help MN. You can apply for legal aid services, you can access the internet. Like I mentioned before most of the things that you can do on a computer. And for Model B you can do the same things but because it's in a private space you also have the ability to conduct a zoom meeting in privacy. You can do that on a Model A as well but it's not really the best experience because it's it could be in a mall for example and also printing and scanning. And so the main differences here are the furniture and then the amount of privacy that you have and printer and scanner. So when I say furniture a Model A has a medical cart that you can use standing up. It also goes down but the sort of default mode for about lack of a better word is standing and for a Model B it's sitting at a desk which also can go up and down for accessibility reasons. So you can see here the difference the person on the left here is using a Model A it's on a medical cart this person is using it standing up and the person on the right here is using it sitting down. You can see the printer just to their right it's a desk and both of them do go up and down but just to give you an idea of sort of the difference you can see the one on the left here is in a lobby and the one on the right is in a private office space. And this is some variation of what they look like out in the wild. All Model B's you can see here they have printers they have some privacy screen setup they have all kinds of information set up but they're these are really the differences and how they look we call it out in the wild. You can see here we have some support information which I'll go for in just a second but this is what they look like when they're being used you can see somebody's accessing Law Help Amendment the Stateway Law Help website somebody here is filling out a form so this is really how they're used in practice. And so if you want to learn more about the project you can find more information at legalchaos.org you're also more than welcome to contact me at triana at goatjtech.com I'm happy to answer any questions you have or talk more about the project but I did want to actually transition to showing you the project website so this is legalchaos.org it's the site that I mentioned and there's all kinds of information here if you're curious about the project you can share feedback if you have any there's a little promotion video that explains how the kiosk work and why they're necessary if you're interested if you're in Minnesota and you're interested in hosting a little kiosk we would love to partner with you so fill out our interest form our locations page here you can find out more about the different kiosk in the in your neighborhood and I have to reload this page actually but essentially this will show all of the different locations and we just have a lot so it takes a minute to load you can filter by a model a or model b and you can even not all of them have this but for some of them you can reserve online um so you can click reserve here if you have a particular location that you know you have let's say a court hearing next week at 10 a.m you can click reserve and basically reserve that that spot so also new automated reminders you can reschedule so we have that functionally not for all of them because it depends on the host site but some of the legalchaos locations do allow for um online reservation otherwise you can call to reserve for example so you can kind of see this is where they are you can click on them it's interactive we have none right now and then upcoming events we have plenty of training sessions if you're curious about how they work you can watch these videos and um all kinds of support resources for users and partners alike I'll just show you the toolkit if you are curious about the you know sharing about the project if you live in minnesota we'd love to have you you know share on your social media you can have posters with some quick guides and support sites so that's it for me today I don't want to take too much of your time but thanks so much for having me and yeah feel free to reach out again here's the URL leoks.org and my email triana at goatjtech.com I'd be happy to speak more with you if you have any questions and sorry I can't be there in person to answer questions live but thanks so much Li-Jie for organizing this and yeah I take care all right I also wish you were here because I just think this is such a very cool project and uh I wish you could be here to answer questions but I guess uh if people have questions or things that they would want to discuss we could have a little discussion or we could move on so if people have questions or comments jump in all right well it's good because we can't don't have answers anyway so that's probably for the best all right I really do just think that's such a cool project and encourage people to explore the website and if you do have questions to reach out to Triana or any of the folks in Minnesota you know to hear how it's going and what they've learned I have a feeling that they'll probably be doing some additional presentation for the legal aid community you know a year or two in to kind of give that kind of information so if we have any of that if we hear of any of that we'll make sure it gets posted to the LSNTAP let's serve okay so I am our third presenter feeling a little nervous which is hilarious okay I'm just trying to do too many things okay so all right you can see my screen okay so I'm going to talk about what's next text this is a project that Michigan Legal Help started a couple years ago and I think David I'm going to put you on the spot but I'm pretty sure all three of these projects that are highlighted were TIG funded projects I know that what's next text was I'm pretty sure that that Lake Shored's video project was but this was a LSC TIG funded project that had two goals we wanted to engage in automate automated text messaging with clients not clients I'm sorry with visitors to michiganlegalhelp.org to collect information about their outcomes for their legal cases but also to provide additional information for visitors if they are struggling along the way in their handling their legal problem and so essentially we wanted to create these text message flows that would kind of follow them along throughout the entire process of resolving their legal problem rather than just the one touch at the beginning and then maybe later they would come back to the website but we wouldn't really know that so it's really trying to get that longer view of just learning more about self-help resources and their impact behind the scenes we created we call them text flows we sort of had to create our own language I mean it's and other people have asked me like isn't this essentially a chat bot and it sort of is although it doesn't rely on AI at all it's very much just a very programmed series of text messages using logic to determine the next question based on the prior questions answer so we created these flows in conjunction with a legal services advisory committee to make sure that you know we were that the ways we were asking questions made sense to them and that we were reaching issues that they regularly heard about from their clients we created the automated text flows with branching logic as I mentioned we built the flows in Twilio studio and we use a Google Firebase to send the API calls to Twilio and collect the data and then and then we try to pull the data out and analyze it and that's kind of the phase that we are in right now so here's been our process along the way this is a TIG with the number starting with 19 so we got the funding in 2019 and we spent 2020 and 21 developing it and launching it and now we are in the data evaluation phase so you can kind of see we picked topics that we wanted to cover and drafted text flows we built them in Twilio studio we did in-house usability testing first with our own friends and family because this was of course you know early 20 or mid 2020 and we were not doing usability testing out in the public like we used to after usability testing we refined the flows we drafted more we had to figure out how to get people into this system how to get them to opt in and so we we created some branding and figured out how we were going to like what our on ramps were going to be to get people from content to the text flow then we were able to do some usability testing with actual end users that we recruited through our website so we did we you know by 2021 we were good at remote usability testing and so we were able to do that we launched the flows we did a couple at a time as they were sort of ready for prime time and then kept adding more and we're at a point right now where we are stopping we're not creating any more new flows but we are collecting data and evaluating in part because that is where we are in the TIG cycle we're working on our evaluation report but also because the whole point of this long experiment was to see what we can learn from the information so this is just sort of a sample text flow as we drafted it you know the kind of directions for the programmer are in bold and brackets and the plain text is what the people see on their phones and you can see kind of the logic being laid out here telling the programmer if they say no go to number 95 we also have timing built in so you know if they say no here they get these two texts and then we text again in three days with this plan for court information so this is just like a little behind the scenes you know what it looks like when we draft it this is just a just to show you the fun logic behind it this is in Twilio studio this is kind of how it all gets built and then these are some sample experience from the end end user's perspective so you know these are our questions and then the little bubbles are their answers it's designed to only accept you know a yes no b c y n and if they type a longer answer they get back an automated message that just says we can't you know we can't understand your answer please reply with a b c or d so that's what it looks like from the end user's perspective and this could be you know these text messages could come over the course of a day or over weeks depending on what the legal problem is and what their answers are but it will all come from that one phone number so it would show up you know as one text thread in their phone these are the topics that we included we picked them kind of based on popularity on Michigan legal help on kind of suitability for what we were trying to do and we definitely learned through you know in the first months that cases where the end user was responsible for moving the case forward are really best suited for this type of interaction because they are the most interested they're the most likely to stay engaged right because they are you know moving their case forward and so we actually added an interview for personal protection orders that I forgot to add to this slide so we have some that are you know petitioners plaintiffs we have some that are defendants you know and the difference between whether you are making this legal problem happen or whether you're responding to a legal problem that's happening to you such as kind of fiction does have an impact on kind of your interest in engaging in a tool like this it seems from the from the the feedback and the data that we have showing that that people you know how often they stick around till the end versus abandon in the middle so that's the those are the topics and we also stopped we have three that we're still working on we kind of wanted to stop and assess the data for a while first before we kept going just to make sure you know we didn't want to make any major changes to kind of the length of the engagement or things like that so that's why those three are still coming soon promotion and data collection the way that we got people that we are trying to recruit people for this tool is through our triage and pages so if they use the guide to legal help and they you know tell us you know determine what their legal problem is and then you know get a specific list of tailored resources if one of their tailored resources is a topic that we have a flow for we sort of advertise the the tool to them there we do the same thing on DIY tool cover sheets so when people use our DIY tools to create legal forms if they're doing a legal form that has a matching text flow their top cover sheet on top of their forms when they print them out or if you know if they open it up to look at it has information about how to opt in to the texting system I'll show you examples of those we're still thinking about other ways to kind of promote it we may add it to more pages on Michigan legal help or even put something on the law help interactive interface in case they never see that cover sheet information and we've been promoting it on Michigan legal help and maps social media accounts as just part of the promotion we are in terms of data collection data collection we're using retool to integrate with Google Cloud Firebase storage to customize and evaluate the data and from that point we just make a lot of pivot tables and please don't ask me any questions about the data analysis or technology because I don't know the answers but I can get back to you so this is sort of if you're on the guide this and you reach the end point so if you said you wanted to make a healthcare power of attorney here's sort of the blurb about what's next text we made it always an opt in system rather than an opt out system because you know so many of our users are you know they're they're running their own legal you know they're taking care of their own legal problems and so we don't want to assume that they want this additional help and then here's what the cover sheet looks like this is kind of obviously a template but this is when they if they print out their documents or when they open them up this is their cover sheet explaining to them what to do next and then if there is a what's next text for this topic they get that information there categories the different kind of legal top level legal categories and so if people are involved in more than one text flow at a time that they will be separate so we don't know how likely that is to happen but we did that just in case and so we have just some of our early usage data we have not been able to thoroughly analyze this data I'm down a staff person who is our data analyst and so this is not going to be a super in-depth data conversation I'm very sad to say but we should have more time to look at the data over the next few weeks and months but these are just some early information about the number of people opting in all of these NAs are because the tool was not live then no surprise that Divorce Without Children shot right to the top this is our Divorce Materials account for about half of our website visits and so you know it's of course the longest and most complex text thread but it is still the most popular and then second again no surprise is the answer to eviction that a lot of people are using that we don't we do have quarter to data but again I'm missing a staff person so I don't have that data here we also wanted to look at what part of so 115 people opted in what does that mean and so we tried to put it into context of like how many people saw that opt-in notice how many people went through the guide or did a DIY tool that would give them the information about what's next text and so we we sort of we did the best estimation that we could and like you know these are are very low percentage points of people who are opting into this which again isn't super surprising it's the very beginning of these tools we didn't really do very much outreach around them and we don't know how interested people will be in engaging in this kind of a system and so I put these here just for information again we look we hope that they'll be growing every quarter and it looks like with this limited data that we have they are you know they are growing which is great so let's see so what do we hope to learn from this project in addition to evaluating you know the tool itself like are our text messages clear are they useful do we send too many do we send not enough the main purpose of the project was to improve our services are the services that we provide as a self-help website and also to learn more about what happens to our users to kind of contribute information to the self-help movement and access to justice movement in general so in the first area we're answering this question by noting where people tell us that they need more help or ran into problems so a few examples are in the eviction text flow we ask did you file an answer with the court and we we keep track of how many people's answer D I need more help is this you know is is this an area where people you know want to file an answer and need a lot more assistance than we're providing what other ways can we provide this assistance or are not a lot of people selecting that they're either like yeah I found it or no I don't want to similarly this is actually we got a lot of hits on this one when we say make a plan for going to court for your eviction hearing did you contact a lawyer many people are selecting see I need help finding one now does this mean that they contacted one and they couldn't get any help or they don't know how to contact one I don't know you know one of the questions that we labored over is like how detailed to get in these texting conversations because there's definitely you know a bell curve of of you know diminishing returns of you know if you ask a million questions people are going to just leave and so we didn't delve deeper into this one and just another one more example is our healthcare healthcare power of attorney if they haven't completed their power of attorney we ask what's getting in the way and so we put some options here to kind of learn a little bit about what might be what are the barriers that people are facing the second batch of information that we're trying to get is people's outcomes kind of the the something that we're always look for looking for in this self-help self-help world and so you know we ask a lot of questions like did you file an answer and so if the answer yes or no these are outcomes that we can track if they're engaging about a motion to modify child support have they filed their motion yet yes or no you know if they're in the answer to divorce without children have you had the final hearing each of these questions can give us information about the outcome of their case and so the part of the data analysis that we're doing right now is kind of isolating these questions and these answers to look at their frequencies among the people who are engaging in our text program a couple of lessons that we have learned already this primarily just about the kind of process of doing this is that it takes a long time to program these text flows particularly the really detailed lengthy ones I mean you know if we're trying to walk someone through the divorce process you know first of all the flow is going to take months and there's just a lot of detailed questions and answers and we had to really fine-tune what level of detail we were going to get into later a lower bullet here is that we needed to not recreate the checklists so our website has instructional checklist that walk people through every step and initially we kind of our instinct was just to recreate those as a text flow and then we realized that was way too detailed and we needed to back it up and so we kind of scrapped a lot of what we had done and started over again to kind of tailor the content to the delivery format there were some tech limitations things are slower with more widgets but of course we want more widgets we ran into you know Twilio kind of changing their carrier requirements over time to making to make sure we weren't spammers you know we had to kind of manage expect we want to make sure that we're managing expectations on the behalf of end users as to what this program is and can do for them you know at the end we do ask them to rate that you know their what's next text experience and you know if you ask someone did you get evicted and their answer is yes and then you automatically say how is your experience with what's next text you know we're not tending to get very good ratings on those cases and then of course like all legal content maintenance is more complex than we had at first imagined so we're already you know doing modifications to the text the text flows particularly in the eviction context as everything changes with rental assistance and whatnot so that is all that I have and I would love to hear questions if you have any and I'm also just happy to see you all again because every time I share my screen you all disappear yes Chris did you say 40 to 60 hours for like each text flow yeah wow I think for the most complex ones I think like that's what we needed to estimate that's what we needed to allow for like the divorce text flow yeah it was a shock to us as well but then when we look at the script and it's like you know 14 pages in word and complex logic it makes sense I look forward to seeing your number crunching after all this because it sounds really cool and you just wonder how how people are if people thought it was really useful and yeah I just look forward to seeing the results I know me too I really hope I get a data analyst hired soon it's killing me to have like this pile of data that I can't tell a good story about but we're getting there hey Matt what's up hi can you hear me okay yeah great thanks yeah I had a question about the text the what's next text tool you said that all of the messages will come from the same number when there are cases or have you seen cases where people have more than one kinds of issue and is there ever any confusion about what the next step which issue that the text that referring to so the way that we tried to address that is we have like we have one number for all of our consumer issues and one number for our family issues and one number for our housing issues and so that was kind of the best that we could do if they have two family lot issues at the same time I think it's going to be a problem but we're again like shooting for the bell curve of doing what we can for the most people and and worrying about the edge cases less if they have like a housing and a family issue then they can have those going into separate text threads but if they have like a you know which I know people do like a divorce and a cuss or like a you know to child support matters then it might get a little bit messy so we have not been able to solve that yeah oh that's great I think having different numbers for the at least the different areas that's can be a huge help because yeah if there are multiple court dates or things like that that could just get rough yeah there though there would be no way to do and I don't even know if our system could handle it well I guess our system would probably be fine but yeah it would be a mess from the end user's perspective thanks yeah any other questions anyone got a lead on a data analyst looking for a job all right well if there are no further questions I think we can wrap up a few minutes early thank you all for being here and thanks for attending our summer series on LSN TAP and our we did take a summer break from our regular monthly webinars which this year are all focused on project management so our next webinar getting too fancy here trying to tell you when the next webinar is it's in August we'll just leave it there check your LSN TAP I'll let's serve for more details on the next installment in the project management webinar series and I hope to see you all then thank you all thanks Ange we'll play you out thank you