 Hi everyone. No worries. Welcome to the third webinar in the LSN TAP summer self-help series. In this webinar today, we are going to hear from three representatives of three organizations that have taken their website to a second iteration or in some cases a third iteration. And so they're going to be talking about the process of doing that redesign overhaul relaunch. And some of the important lessons that they have learned along the way. We have Gwen Daniels from Illinois Legal Aid online. Emily Good from Law Help MN in Minnesota, and Daniel Ereger from the Northwest Justice Project in Washington State. So we are going to start with each of them giving a sort of overview of what their transition process was like when it happened or what stage they're in now. And then I have a series of questions that I'll ask them to have a conversation but if you have questions along the way, you should also feel free to put those in the chat or raise your hand and we will let you ask them live. We do ask that everyone mute themselves unless they are asking a question and that will help keep down the background noise. So Gwen, why don't you start us off with telling us a little bit about LAO's transformation. I'm Gwen Daniels. I'm the deputy director at Illinois Legal Aid online. We have been around for 21 years. Our first website went live in 2002 and has since gone through, I would say about five major overhauls, the first one in 2005. Then again, when we launched the online transit intake system in 2013, we had a complete rebuild between 2014 and 2016 to move off of cold fusion and moving on to Drupal 7. That was probably our most expensive and most painful redesign. And in 2019 to 2020, we did another redesign going from Drupal 7 to Drupal 8. And now we are on Drupal 9 and that was a fairly seamless upgrade. Great. Thank you. Emily. I'm Emily Good. I'm a project manager at Legal Services State Support Minnesota and we run a bunch of the websites statewide that the big one is lawhelpmn.org. And our site originally launched in 2005 as part of the Pro Bono Nat Law Help sites platform and we were on that platform up until we did a redesign of our site that started in 2018. Well late 2017 really, but mostly happened in 2018 and then we launched a new site in 2019 and we, during that process moved from the law help platform over to Drupal. And there were a couple of factors that made us decide to make that transition. One of them was wanting greater control over how we put content up and what sort of editorial control over how the site looked. Another really big piece was as a lot of folks were sort of pushed towards triage having a more interactive feature to help clients identify their legal issue and match them with resources and referrals. And wanting our online intake to the different programs, legal service programs that we work with to flow more seamlessly out of the law help out of our law help site. And so that was sort of what motivated us to decide to work and move over to a Drupal platform. And since then we've made a number of iterations in the site, we haven't done another massive overhaul, although we continue to make tweaks and changes and all of the programs that we work with, for example, are now on legal server and so our online intake integration matches up with legal server in a way that it didn't when we did that initial move over to online intake. And that's been one of the great things about the site that we're on now is that we just have a lot of nimbleness to continue to make those changes and developments as things happen. Talk more about kind of specifically what those look like but that's where we are at. Thank you so much. And Daniel. Yeah. I always think I visual visual aids help me a lot in learning. And so I'm going to share my screen to show sort of, let me know when you can see it. We can see it. Okay, do you see the PowerPoint slide. 1.0 2.0 3.0. Okay, so this is the history of Washington law help.org. So it started in 1998, and we had a quote in the sort of script from judge learn at hand. It was basically a combination with NW justice.org. I think this is before my time. But very text heavy. And, but it was, it was that's the first attempt to present legal information and sort of plain language and it's kind of like a digital brochure rack. And the 2.0 site launched, I believe this is the pro bono net platform in 2006. And we since did a redesign, we call it 3.0 in 2016. And this slide is actually from a presentation that I gave last fall so it's a little outdated because we're honestly on I guess what I'd call 3.5. That you can see it looks a little different we got a refresh the pro bono net platform rolled out a refresh look. We got a new logo I like this news more streamlined logo. And you can see it's looks kind of the same, but it feels a little more 2020s type style. And, but you have 4.0 here because we applied for a technology initiative grant from LSE in 2019 to redesign our site make it more accessible, easier to navigate mobile first. And we are right in the middle of that project so we started last year. We did a massive inventory. Oh, I should show you this to might be interesting. This is another history. We were getting about a quarter 250,000 visitors when we first launched and we like, we now have about 2 million people visit the site a year. This is like pandemic era but a huge increase during the pandemic, and we started doing webinars and a lot of other things. And the laws in Washington for tenants changed in a big way last year there's now a right to an attorney and eviction proceedings for people with low incomes, and we have just caused eviction it statewide so that meant we had to do a huge amount of public legal application. So we created dozens of new fact sheets and infographics and videos and webinars, and we've been prioritizing translations into 16 different languages. So it's a lot of just, we've just been making many, many new things. So for the first time project last year we started with a lot of usability testing, so we set up this website improvement program and offered people $20 gift card if they would spend an hour with us share their screen. Give them a hypothetical we actually did comparative analysis with all of your wonderful sites to we really admire law help Minnesota Illinois legal aid online Michigan legal help. And we did listening sessions with law librarians because we know that law librarians social workers volunteer lawyer programs they all rely on Washington law help.org in there in helping people. So it's not just people who aren't lawyers it's people helping people who aren't lawyers. And we, we, you know, some of you may know that there was an outage of the pro bono net platform in March. We took our website down. There was some security incident at the pro bono net side. So our website went down for a week, and our interactive interviews and our forums went down for about a month. And we sent out surveys to people like how is this affecting you and we got a lot of, a lot of, please put the website back up. We need it, we need it now. So that's a little summary of Washington law help. I'll just say, sometimes this is this may be illustrative of the mass of the site. Because we hadn't really ever done a huge inventory of what what all do we have here we keep adding things to keep adding things over the years, but we need to step back and see like well our people are actually clicking on this so the inventory we started was in a spreadsheet that is here. And you can see some of the resources get hit thousands of times, tens of thousands of times, or there's a lot of stuff that doesn't get hit, but there's over like 1400 items here. Some of them are external links some of them are fact sheets. We and we were trying to do the content management system upgrade and that's a big part of why we wanted to redesign the site is because it's for. It's just such amount of material that we don't even know what we all have and we know a lot more than we did but we've since started using a project management program called smart sheet. So we sent things over here but we can now do manage our workflow a little better but we're hoping to improve that process to when we select a vendor, a web developer to redesign the site. There's more to say but I'll stop for now. Great, thank you. Yeah. So, Rochelle has a great question and I'll expand it a little bit and say, when you did your redesigns. What, what things did you user test and did you do that in house or did you contract that out as as to another organization to help or another entity to help with user testing. First, we did not contract that out. We, we, like I said we had this week we called it a website improvement program. So right on the website it said join us and we had people sign up. And we did a couple. I mean we did like, sometimes we give people a scenario like if you received an eviction notice. Okay, somebody told you to go visit this website. What, what would you do to find help. And then we would record them doing it and ask them to. It's called like thinking aloud, kind of like I'm thinking, I should click on this but I'm not sure let me try it. Oh that didn't work. And then we would took, took all of that those conversations that we had, and we dropped them into a program called dovetail. And dovetails for usability testing. It lets you take all of like these conversations and see where people are getting frustrated and tag that as like a frustration point. Or tag it as like easy to find or tag it as like, I'm, I'm wishing there was something else here or oh I like that there's a video and we tag it and so you're able to see patterns that in that way we have a ton of qualitative data that we gathered through that program. That honestly we haven't even had the capacity to stand to do deeper analysis and we're hoping to work with the web developer that we're going to hire to do another level of what do we, what's working and what's not. I think we had, we had practice at doing stuff like this because we had received a technology initiative grant in 2016 to do usability testing of videos. Because we have been making YouTube videos. And, you know, people would ask a lot of times like skeptical lawyers would ask, do these videos actually help people. And honestly we just didn't really have like any evidence. I mean we don't know people would anecdotally say oh I like that video and you made it really clear, you know how to ask for repairs for my landlord kind of thing or what to do if you get sued by a collection agency. But we actually went out into the community and did usability testing sessions so facilitated a discussion. We learned a lot about how to do this from an organization in British Columbia, and so short answers we did it in house. And, and like I said we did other things called listening sessions and that's more just like getting in a zoom and having people tell us about their experience with the site like with like librarians for example. And we learned things like, like in certain libraries, like they are, they've pre printed out this sort of like Washington law help. Like, here's where to find stuff and then they'll write in. Like, when people come in and they need papers for like a parenting plan. They'll write in here's what to look for and we, we, that led us to the idea that like oh maybe we should create those for people. Like that sort of print resource or something or maybe it should be a print here kind of thing. So that's, that's this short version of the major efforts we've made so I'll stop. Next, we've done a lot of user testing over all of our iterations and it's varied, depending on when and where, when and what we were testing. For the last several years we've done quite a bit of user testing using usability hub, sometimes with our own panel sometimes we will pay for their panels because we can get results back really quickly, but it tends to be more of a test and you know first click design preferences those kind of tests. We also did a, because we completely rechanged the site architecture in our 2016 rebuild. We did a lot of card sort and tree navigation tests online. We used to do observation tests to do it now more over zoom and in person that we did do some in person. We had staff that would go out and stand in the street pointers pocket and stop people and ask them to do a quick user tests on paper and give them a gift card. And we do it at different points in the process sometimes we'll do it when we have an ugly paper prototype sometimes we'll do it and we have a polished design that hasn't been implemented, and then sometimes we'll do it once the it's been implemented and it's on the website. And I'll talk briefly, we did, when we were actually doing our redesign, there was some user testing that was built into that and our, we work with an outside firm who did all of the technology work on the site. And they did they ran the actual testing but we were responsible for coming up with scenarios and finding the people to show up and and just in total honesty I think for us sometimes it's been really hard to find people and to get people to show up I feel like when people talk about it like oftentimes I feel like oh like you guys seem to have no problem getting people and we often struggle to get people to show up for user testing. So I don't have any brilliant advice on that. But we've offered gift cards and done a variety of things. But we've usually done our user testing in partnership with somebody else. When we've done some more recent user testing, we have done online through different and I wasn't in charge of that so I can't remember the name of who we worked through to do sort of some of the online usability it was related to accessibility testing. But that's been somewhat more successful, but we don't do. I would say, super consistent in house user testing it's something that we do in when we're doing specific projects or rolling something specific out. I think we honestly struggled with sometimes to do and so we rely also on looking at Google analytics data on looking at search terms on getting feedback from our partners and other users. We have also worked with librarians and other sort of trusted intermediaries and we did some focused testing with them when we were rolling out our redesign where we had small groups of folks come in and navigate the website and answer questions and give us lots just in an interview and that's something that we had a consultant through a separately funded project who helped us with that, but we again we're responsible for finding the partners, and for coming up with the scenarios, and then the consultant helped with sort of structuring that and managing that feedback. And that was super helpful because it gives you of course a sense of what these power users are asking questions about and also things that seemed obvious to us, but like a social worker was like I don't understand why I didn't just get a list of lawyers out of this and we're like, did you think you were going to get a list of lawyers out of this but that's still really useful information to have. That's what we've done, or not done for user testing. I'll just add, I'll add one more thing because Michigan legal help we're in the middle of our own Michigan legal help 2.0 project and one thing that we did that was really helpful was we got a usability audit from the graphic advocacy project. We put together a whole report and slideshow of like, you know, suggestions that they had for new layouts or, you know ways that usability and accessibility can be improved, and like having that document to give to our developers was really, really helpful. So on this on the topic of sort of, you know, you're doing usability testing because you're doing a redesign can you talk about some of the major changes to design or architecture or functionality that you made during these transitions. So the biggest change that we made when we made this shift was we had not had any sort of triage. What we had prior to our shift was a library with a lot of resources and as Daniel was talking about like, we had resources that we probably didn't even know we're in there. And then we had sort of an organization list where folks could search a static sort of phonebook style directory of what legal service providers were in existence. And the biggest thing that we did was to smush those things together into a Q&A format into a triage system that we built that would. We relied a lot on looking at what other folks had done. So like looking at how Illinois was working looking at how Michigan's was working looking at Connecticut and Pine Tree I believe also at the time, and seeing like how did how were questions structured and like one of the decisions we made is that in our system, you, the first thing that you got were resources and then you could answer questions about your location to get a referral. But we sort of one concrete decision we made was to separate that differently than what we had seen done so far. But then it also meant that our some of our logic flow and other things had to shift. One thing I should mention about Minnesota that may or may not be unique but we have seven, six legal services like LSC or legal services legacy projects and then we have probably 12 other legal services organizations and they're all independent entities but so we don't have a single program that we refer to we have all of these different programs that we are trying to work with and coordinate referrals to and so part of what was involved for us was also just like making sure that we had good info because the other thing we were shifting was from a single point of control is that we wanted to make it possible for organizations to edit and control their own listings so that if are the office that's located in St. Cloud Minnesota got a grant to do crime victims work that they could add that information to their own listing and update the types of cases they were taking or that they could say we are our office in Grand Rapids just lost the only family law attorney so we are closing intake on family law for right now and part of what we wanted was to have that information about like what services organizations were actually providing more up to date than it ever had been because we knew that that had been really static and so we have a way in for the organizations to edit their own listings so that that helps with that referral and intake process and so that was the other piece we were adding that was unique and different from what we had had before and now we've added to that to have like our staff directory which is only available to legal aid staff but our staff directory lives on our site now to because it's based on a lot of the same information so hopefully that's kind of a useful but and I'll I know there was a question in the chat and I'll just answer that and I'll hand things off to Gwen or Dana. So in terms of Illinois I think our biggest I mean our biggest really was going from cold Asian to Drupal 7 and then from Drupal 7 to Drupal 8 because everything had to be completely rewritten from scratch. We have never been on like the pro bono net template or another tip that we've always rolled around. And which meant I wrote every line of code in the in the Caucasian website and that had to be transferred into Drupal 7 and while there was a lot that we could do off the show, there was still a whole lot of stuff that had to be custom built we ended up when we launched I think we had about 50 to 60 custom modules in Drupal 7 or Drupal 7 to Drupal 8, all those custom modules had to then be rewritten into Drupal 8. And we had to do a whole new front end because the templating language for Drupal 7 and Drupal 8 are completely different. We also, when we went from cold fusion to Drupal 7 we also made the decision to rethink how we get all of our legal content. And so we created new content models and we had to migrate all of that content over. And similarly we had to rebuild our online development and take systems that at the time supported six organizations and now it's up to 12, which has also just gone through another major iteration there. That's not visible to most end users. So, I guess the question was what changes did you make based on the testing. Or just generally, you know what were some of the major shifts, major changes. And if you don't, I mean I know that you're still very much in progress. So you can take a pass at any time. Well, I mean, we, yes, we are in the beginning of the process and the web developers, whoever we go with the, there is going to be another like discovery or analysis phase at the beginning of the design process, where they're going to help us analyze what we've already collected and then also do more. That's what the companies have told us. So, we'll be finding out more but like, I think certain little things were confirmed, like things that we suspected like, there's two channels we call them in this platform that we're on right now like you have and then right next to it is this tab that says court forms and procedure. And, like, people don't see that very well and they don't click on it, you know, they're just like if it's not here then there doesn't seem entirely intuitive like well then I'll go over here, you know what I mean. And so, you know, duly noted, that's a change we could make in the new new site, you know, there was another thing that there was just this like sort of like the way the table of contents thing and it was this module that we couldn't change and it was like, if we were linking to an external link, but there wasn't much on the page, it would like have the table of contents it would say go to link, and then the link would be there. So people would click on go to link. The page would move a little bit. And they would assume that the link was broken. And then we actually told pro bono net about that they changed it. Because we saw from the inventory to that like, wow, people are landing on this page through Google search like Gwen noted most people come into the site not from the homepage but through searching for eviction Washington state or something like that. They would go on to a resource. And, but we saw them coming to those pages where we were linking to like, I don't know, the US bankruptcy court, and then not clicking out. Like, they would just get there is not. So anyways, that's, that's improved. We've, there's other things that we'd like to have more obvious or easier to understand like, you know, if you get to that thing that's adjacent to what you actually are looking for. We want to make it more easy to know like what's, what are the realms of things that are over here, you know, because right now it's there's a lot of like it's still a little bit feels more like that brochure rack and if you don't know exactly what you're looking for, if you get to something right next to it, you might just leave. And we've no, we've been looking at all the others sites and one thing I noticed about West Virginia site is like I love how you can you look at the phone. And there's like all these different topics like eviction housing subsidy. You know, and you can click on more than one and it'll just filter and kind of narrow it down in that way and it's, it's a good. We've noted that that's like something maybe we want to have. We were looking at other sites we didn't have an accessibility tool like a thing. We have it now where you can change the font, the text size you can, you know, do reverse contrast and we didn't have that and it's, it seems like, oh well, an obvious improvement. But it is something that we have changed after noting like, oh, they have that we should have that because that that'll make the site more accessible. And it's called user way. So if you go to Washington law help you can click on this and change how the information is presented. Great. Thank you. I would love to hear from each of you. An interesting challenge or unexpected issue that you faced pre launch so in doing the work. You know, some something unexpected that happened and how you handled it or resolved it or what you learned from it. Yes, I could go when we did the co fusion to Drupal seven. We build and I did a session as one of the LSE conferences a few years shortly after that called how not to build a website. And it was took twice as long as we expected it to take the almost burned out the entire staff. And cost twice what we expected it to cost I think we were somewhere in this around $750,000 for the read design rebuild in terms of staff costs and time and developer costs because at the time I was doing most of the development work they did bring in some some additional developers to work on it. One of the most important things that came out of that process is I no longer do development work. We've actually I don't scale up enough and so without source all of our development work to a development company out in India. And we have three full time developers who continue to work on the site year round. And probably the most important thing to know when you're building a website is you're not done when it launches. There's always something else to do. Great. Thank you. Yeah, I think just kind of, we were building out a triage guide but our developer had never done exactly that before. And so, working with some of the they had outside contractor helping with a part of it. And I think just it turned out okay. And a lot of the communication between we had a couple of staff members who really didn't have any technology background which is fine. I don't really have a huge technology background but we sometimes spent a little bit too much time in meetings with the developer, trying to get everybody onto the same page and that ate up a lot of time and resources that we probably were spending that way and so I think making sure that you have the right staff people in the right positions, who are, you know, making decisions and having everyone on the same page about things but not getting bogged down in sort of, don't get stuck, I guess I would say, because it's easy to have that happen and you want people to understand what's happening but also know who really needs to know, and what they need to know and when. So that you can keep the project moving forward. And that I think just continually having the person though who did have really great tech skills and ability to communicate that making them the point person with the developer on some of those sort of more tricky pieces where we were trying to find what the needs were for example in our triage guide and making sure that the developer really understood and making sure that we were reviewing those prototypes and going, oops, this is not what we thought we were getting. Making sure that that feedback cycle is quick so that you can continue to like stay on task and stay on time because we actually pretty much came in here we thought we would time wise, but it was a little bit of a struggle to get there. And I guess, I mean, now that we're in the pre phase, I, I don't know if it's a challenge, I guess it is a challenge but it's been, it's been an education for me in what is a content management system. So much the difference between a headless content management system, and, or a disconnected or uncoupled one. Those are terms that like maybe I've heard in my life but like, I don't know I was focusing on trying to do legal education stuff and I didn't pay that much attention apparently but but in fact it's a necessary component of a website library this vast is like, we're, you know, you have you have a CMS content management system whether you know it or not, it just is it the most efficient way that you could be doing all of this like we you know we pass along Word documents with track changes to, you know, volunteer communities and then give them to our, we have two coordinators and then it gets formatted into a PDF and it gets put up on the HTML and there's this 30 step checklist about how things flow, but right now. It's kind of at that spreadsheet level and we're just like, Okay, which ones is who's working on this one. We want to move to a different kind of system and that's been a big part it's really like to improve our capacity of our public legal education attorneys so we were able to respond more quickly when there's changes in the law or like when and make more timely items because right now it's just, it's, we have a we have a pretty. It's not an efficient content management system where it's not as good as it could be and that's that's been a big challenge that we're going to be working through with the web developers who all have their own preferred ways of doing that. We've learned about different products and solutions to that and we're going to be working on that but that's just a big thing. A big part of a redesign project that I wasn't totally aware of last year and now I'm like okay now I really, I'm excited to get this new CMS so I can. Like, when something changes like the amount that you can sue for in small claims court, or like an eligibility guideline and it's like, it needs to be changed in like 100 different places. And right now it's, it feels like we have to manually go in take the thing down put it back up. Okay which one is it is it over here and I want to be able to like make a change to this one type of micro content, whatever it is. Somebody's phone number and organization so number and I wanted to just keep ripple across and I'm hoping we move to a CMS that's more like that. So I just threw your through your comments. It sounds like with the exception of LA O's cold fusion to Drupal transition everyone has hired has outsourced the bulk of the development work is that true or planning to hire planning to outsource. Yes. Okay, great. I just wanted to just sort of make that point because I think in Michigan is Michigan legal help has also is also done the same has also done the same thing both the graphic design and the development work because none of us are designers. And you know while we technically have people who can do some of the development we don't have the time we don't have the scale that we need for these types of projects. See your comment. Thank you very much. We've already been learning from you, because of you, the way you've publicly allowed the public to view like that how you have different types of content. And I, you know, when I, the word is so vague I just like it's one of those words that my brain just, what are you saying content like it just seems like it's everything but but okay we're talking a title. Oh, that's right every resource has a title and then a number and then a difficulty of the legal problem might be another type of content, whether it's a legal problem or illegal solution, whether there's an organization that's available. You know, all of those are different little types of content and to having to identify what you already have where we're right in the middle of that process. It's great to hear about unexpected issues and challenges pre launch and I guess this is mostly for Gwen and Emily. What about post launch, or some of the big lessons there. We didn't realize exactly how hard we were going to be hit by bots. So first couple of days of the size going live. We're going to have these oh you've got all these new pages we're going to hit rapid fire hit the website and you know reindex your site and being doing the same thing and so we had a lot of performance problems the first couple of days. Luckily our developers were able to go in and do a few things under the hood to better manage some of that. But we had a first couple of days were kind of painful. We had a lot of redirects, having to redirect the old website URL to the new website URLs and helping things didn't get lost. And we thought we'd we thought we had done it all right and triple and then turns out no, then it didn't work that way you have to actually go in and edit the raw ht access file on the server to actually do the redirects before it hits triple. So that was a big challenge. I also highly recommend Kate and champagne for every website launch with the whole staff. Yeah I think I would agree that like food is a good reward and fortunately we didn't have sort of those technology glitches in the same way. But I think we expected and this actually come up in a question about like triage. I think we thought triage was going to get a lot more use than it did and has. And so I think there was both excitement. Maybe a little more existential but like I think there was excitement about the amount of time energy and effort that had been put into that and then it was kind of like what are we not doing right or what is the site not doing right that these numbers are not higher than they are. And that is something also where we've just, that's a nice thing about being able to continue to make tweaks and changes is that part of having that control over our site is being able to devote time, money and resources to saying okay let's try this as a way and also building out our Google analytics to be able to track what's happening through each of those pathways. And so as part of the revisiting things continually. But also that continual what you think is continual improvement. I'm just takes kind of what Daniel was saying about content like how much content there isn't just how much time it takes to get that stuff up and going and that for us like how much time it takes to revisit our triage questions and review those means that it really is only going to happen once or twice a year, because it's a massive staff time undertaking when we do that. And similarly, our, we've always had sort of a content management system because we've had this group of fact sheets that predated the website existing which is great. But like the amount of time that takes and we have them all in multiple languages and I think as we built out our language resources, that's been great but the amount of additional time it takes to keep things up to date as you're talking about. If we change wording in something or something changes in the law there's three other languages in addition to English that we need to go make sure that that's been updated in and I think just for us I think continuing to be really honest that everything is going to take more time than you probably think it will or should, even if you're good at it. How did you handle PR about the new site with lunch did you do like a soft lunch and then do PR did you do it all at once. I'm covering my eyes because we, we've done it both ways, the soft lunch is much better. That's when you do a press release saying this live and then you suddenly get hit with all the spot traffic, it's really unpleasant. So yeah I highly recommend doing a soft launch and then doing the announcement to everybody. And that's also what we did. And a lot of our partners knew like our legal aid partners knew it was launching but yeah we didn't do a push, and we just reached out to sort of local media. One really sort of funny one that we had as we were in like the, I think with Minnesota doctors magazine or something as part of like healthcare legal partnerships and they had us do is actually super cool we did a whole like tear out in that magazine. That then doctors could like put up on their wall that was like here's how you can use this site when your clients are coming in with questions. So that was like, I think some of those sort of surprising non legal networks were helpful. And then we leverage like a lot of our legal aid and other partners to say, Hey, make sure you're telling people that we have this and just sending out a lot of materials when we relaunch to. Great. Did you come across any tools during your redesign efforts that were particularly helpful like, you know case management tools or, you know, anything I feel like, you know Daniel mentioned the accessibility tool. You have any recommendations for, or tools that were useful in planning or executing this process. We, we relied and so we like very heavily on giraffes. And if you're not familiar with it it's a product that is really was designed for software developers. And it allows us to create agile background stories, tickets, and then communicate acceptance criteria and move things across a development process. And that's something that I still use every, every day with my development team. I think I said thank you when I think I said case management systems I do met I do mean project management systems. Daniel put a link in the chat thank you. Yeah, that's that's dovetail that's so that we use to gather the qualitative conversation kind of data and to be able to tag it to see patterns. The project management system that we moved to we were we were using something called base camp. And we moved to smart sheet, and I like smart sheet a little better because you know we've got it first off we have a, I guess I called a spreadsheet with like 1100 items on it, and all of these different columns. And, you know, we have, well currently we have five public legal education attorneys so every one of those resources is assigned to one of those attorneys who's the primary editor. And we're actually going to be losing two of them. So we just recently had to slide all all the the cards because you can switch from grid to card view. And you can also do Gantt view and calendar if there's dates involved and we really like that feature. And in the card view it's easy for me to see all of these are in your bucket. Okay, well we need to reassign them over here. And it was like visually and the way it's used is it has more function than we were getting in with base camp so it's called smart sheet. And then you can connect the sheets. So like, you can have that's the everything's based on this major one and but then like if somebody, we can make it to do list for like one of the coordinators by once, once their name gets assigned to it or tagged in it then it shows up on their task list or to do list. So if we're just doing like software roundup. The other one that we've recently got and not quite as part of the redesign but maybe we started doing webinars. And we needed to make transcripts. We really we needed to make caption files, but to do that. It's a similar process and so we, we got a program called Trent. And you can take the video file like of a recording, and you just drag and drop it into this interface. And within seconds or minutes it will generate a transcript and it will recognize different voices and so you can go in and say oh that was Daniel. And it'll like ripple and try to run it's not perfect you do have to have somebody go in and, you know, tell it, but you can remove the thumbs and the Oz. You can capitalize Washington law help and then it'll like okay do it throughout, and it also generate a caption file so that's good for, I mean that's what you need for accessible videos so we've been using that a lot. And I'm also other. Oh sorry, you can go. I have a couple other things that we use. We use a product called Azure and CRE for doing rapid prototyping. Because it lets me do drag and drop widgets on a page and I can sort of mock pages up before I have to put them in front of a designer to make them, you know, pretty and give me something to test off of. So, if you install the developer plugin for Chrome there's a tool from Lighthouse that will allow you to do performance and accessibility audits on a one page, and it's free. And it will give you at least an overview of how your page is doing. There was a question in the show. Sorry Emily did you want to did you want to know I was just going to answer the question in the chat. And so I'll put it up to everyone. The question is about what kind of staffing you had, I guess this is in house because the, you know, you all have hired outside developers what kind of staffing in house did you have and then, if anyone has, you know, a budget for the, the, you know, the, the process of creating the new website and Michelle points out we know that the cost don't stop there but like for this phase the redevelopment phase. So staffing and costs. This staffing for us. At the time that we started the redesign, I think we were a staff of our staffing has changed a lot. And because of COVID I'm like who was on maternity leave when I think we had five in house staff members, three of whom I would say, there's a single 10 who a lot of people know me and then our sort of office and director, who we went through a shift in that right at the beginning of the project which was hard. I'll say that. But so the three of us really sort of were in charge of everything we had the other staff members actually six total I should say. So staff members got pulled in to things like as needed like when we were migrating content. That was sort of an all hands on deck everybody helps upload resources and add resources on to the new platform because there was not a really easy way for us to move things over from the platform to our Drupal platform. And, but so that's, I would say three staff members who were pretty much we did have other things we were doing at the time, but not a whole lot. And the redesign with the biggest piece. I'll just say that the firm, the development from we worked with was a pretty small shop. So, less do you think that there's some sort of behemoth. We had, you know, they're a four or five person. So that's the role including like the designer and everything else. And then cost wise I just pulled up one of our budget sheets to see it and it looks like we were right around 250. And I, that doesn't include but I'm pretty sure we got some in kind hours donated from our development firm if I'm correctly recalling. So that's at least sort of a snapshot and like I said that doesn't include the maintenance or anything like that. I don't want to answer this. So, I mean, I mean, I mean, yeah, it's always been very fortunate to be well funded and both staff. And I don't know what our, what our budget for what we are actually spending sure which for this last three design, I can tell you that my maintenance that my annual maintenance budget for the website for the developers is about 270,000 a year. Which is about 12 or 13% of a leo's budget. And we have four full time content members and a product support manager who is just doing Europe's and online triage and it takes system product manager. And then I'm still doing about half, probably maybe half of my time managing the developers. Three full time software engineers that we are on contract and a half time quality assurance analyst and 10 hour a week product manager project manager of the development team. So, so I don't want to do that. Like I said, it's been very fortunate. Our team has expanded and then now it's kind of contracting but and during coded it did this so like, we have a website manager Danielle rebar who has been maintaining Washington law help and also nwjustice.org and our staff internet for 25 years. And she does the work of probably three people. And we have two coordinators, so they do administrative assistance. And we, we had, we had one at first in during the pandemic and then we've gotten another full time coordinators and then the attorneys, we have one attorney who is just project managers Washington forms online. We have two interactive interviews. And then we've gone from, from, let's say point four because I was, I was like a part time working on Washington law help making videos when I was also working as a staff attorney on our, on our statewide hotline. And I did that for many years. Then I became full time part of the team. And while the Washington state provided temporary legal assistance money through coven or in response to coven and so we were able to get two more full time attorneys. And they were specific like one was focused on language access one focus on eviction and housing. And now we're going to be at three, three of those attorneys including me. So what is that 1234567 the Washington law help team is a team of seven. We don't have a developer. So we're going to have to rely on the developers and our budget remains to be seen but I can tell you what we put in our RFP, which I'll drop a link to the RFP we should probably, it's still up on the website. It's still good proposals. But that's just an example but that was a whole challenge in a process in and of itself is to write that RFP and we learned a lot by meeting with Illinois legal aid online and law help Minnesota and Michigan legal help so but it's based on a federal application for a federal grant it's a tick grant so that that the budget that we put out there is 100 and 150. That's the range and we know there's going to be ongoing maintenance costs and we will see how much. All right, thank you all we are close to the top of the hour so if any of you have like one, one really important tip that you would pass on to anyone in this position. What would that be the one tip that I read actually in a design about design generally, which is that in any system there's a fixed amount of complexity. And you just get to decide who has that complexity, and sometimes like if you're trying to save money. That may mean that the user has a little more complexity, because spending the money to reduce that complexity is going to be bananas. It might mean that that complexity goes to your staff who is going to maintain the thing but that's that was super helpful to me as a decision making framework is that there's a set amount of complexity and you just have to decide who's dealing with it. That's great. And for me, I would say the most important thing is to make sure that you're also take that you're taking care of yourself while you're going through these three designs because it's a huge amount of stress. My content team will also tell you to remember that they're users to do in terms of configuring your content management system and back in tools and reporting affects them more than it's going to affect anybody else and they're the they are your heaviest users of the system. So kind of keep them in mind. Yeah, I don't know. I don't know if I have any big hot tips yet. We'll come back to you in a year for your come back in a year. Awesome. Well, thank you, Gwen, Emily and Daniel so much. This is a great conversation. And I appreciate everyone who's on the call for coming to listen and add questions and things. The summer self hop series continues. Next week, July 14 will be talking about user centered design and self hop websites with Ashley 20 from the graphic advocacy project. It's a great series of the dirt to put together and looking forward to to hearing more next week so thank you all so much and have a great rest of your weekend weekend.