 Here today we have Glenn Rodin who is the Program Counsel for Technology at the Legal Services Corporation. Glenn is responsible for helping legal services programs with their technology efforts and the administration of the technology initiative grants or as many will call them the TIG grants. Since the program started in 2000, TIG has made over 640 grants, totaling over $53 Many of them in partnership with SJI and the state courts. Glenn is a member of the Executive Committee of the Self-Represented Litigants Network and a frequent speaker on self-help strategies. He is a fellow in the College of Law Practice Management and before coming to LSE in 1999 he was a managing attorney at the Legal Services of Eastern Oklahoma for five years and before that he was in private practice. He has served as co-chair of the Office Management Section of the Oklahoma Bar and he is also a member of the Legal Technical Advisory Council for the ABA. That is all and I'll let you kick it off Glenn. Thank you very much for guest lecturing this lecture today. Well, I'm happy to do that and I want to thank you all for participating in the program. We've got some wonderful opportunities to work with our programs and legal aids on helping them. I thought I'd give in case people don't quite understand what the role of the Legal Services Corporation is, LSE is a federal nonprofit. We were created in 1974 by Congress and have been refunded every year since to provide grants to independent nonprofits around the country who provide free legal civil legal aid services to poor people, people who are under 125% of the poverty level. Currently we have 133 programs that scratch all the way from Puerto Rico to Micronesia and so as you can imagine with large territories and limited numbers of attorneys it's very important for us to learn how to be more efficient and effective in the delivery of those services and then to be able to provide services so that people can help themselves when we're unable to help them. Unlike many programs where if you are eligible for the program, you are entitled to receive the benefits, LSE's legal aid program is not like that. You might qualify for legal aid and still be turned down because of a lack of resources that the organization would have to serve you. We spend much of our time in doing this so that we can provide services to the people that can help themselves and the session today is called Changing the Landscape of Legal Services and I wanted to give you a little idea of why that's important. If you look at an operating room from the 19th century you'll see that it's fairly stark and about the greatest technology that you see there are the gas lamps above the table. If you look at a modern operating room in the 21st century you'll see lots of technological innovations that help the surgeons as they go through their work and perform their duties. Now if you look at a 19th century courtroom on the left you'll see that it looks very much like a 21st century courtroom on the right. The one on the left is from the Old Bailey in England in the 19th century and the one on the right is a renovation that was done to a courtroom in Tennessee in 2008. I think it's a good idea that we embrace our past and I like seeing things being restored to how they used to look, but at some point we're going to have to actually move out of the 19th century when it comes to law and get into a 21st century approach and I don't think that we really have been doing it all that well and so that's one reason that we talked about changing the landscape is that we've got to move into the promises of this 21st century. LSE had a technology summit a few years ago and out of that summit came a very important mission statement which has now been adopted by LSE's board of directors and that was to explore the potential of technology to move the United States towards providing some form of effective assistance to 100% of persons otherwise unable to afford an attorney for dealing with essential civil legal needs. Now you'll notice one interesting thing about this mission statement is it doesn't say 100% of low income persons. LSE recognizes that while our mission from Congress is to help low income people that the United States as a whole needs to be helping all of the people to achieve access to justice and so this was something that was very important that came out of our report and at the end of this presentation I have a link to that report so you'll be able to see it. Now this was very much repeated by the Conference of Chiefs Justices and the Conference of State Courts Administrators in a resolution that they passed in August of 2015 and as you can see they have adopted the same goal of 100% access to effective assistance for civil legal needs for all members of the United States and they're urging all their members which includes every state court in the United States to actually adopt this mission statement and to move to 100% access to justice. There's an initiative that's being funded by the Public Welfare Foundation called the Justice for All Initiative and it is working with several different states around the country to actually do the strategic planning necessary to achieve this goal. The self-represented litigation network which LSE is a member prepared a bunch of documents to help the states in the strategic planning and this is in conjunction with the state with the National Center for State Courts and working with the Public Welfare Foundation that this planning is currently underway in these states in doing a strategic plan that will help the states map out how to achieve 100% access. We're hoping that they can lead the way in showing other states how this can be done but you have to be very strategic about it and I'll talk a little bit more about that. Out of the technology summit we came up with five different incredibly important initiatives that we could use. Now as you can imagine we had four days of meetings and in the first two days of the meetings we came up and kind of brainstormed. Let's say that money was no object what would you do to improve access to justice and to think outside the box. And so for the first two days of the summit which we held in the summer then we came up with that brainstorming with different methodologies and such that we could use. Then we spent the next several months using some online tools to help us prioritize those and when we came together in January of the 2013 we actually decided that we had to come up with the five most important initiatives that could help move access to justice forward using technology. The first one of those was using more use of automated forms and other documents. As you know we have a system of justice that's based on forms and pleadings in our courts and it's very important for us to provide the tools for those people who are trying to help themselves to actually fill out and to get these forms right. Oftentimes the forms are not in plain language and this can be very confusing to people. We have a tendency in the law to talk in legalese to have our own jargon and so what we want to see is more use of the automated forms that can step by step set out these questions so that it's easier for people to fill out the forms. And I'll show you a little bit more about that later. Now one of the things too is that and we'll talk more about this a little later too. The use of technology, mobile technologies for communications is getting more and more ubiquitous. And so it's very important that we take advantage of the promise of the mobile technologies. Also we want to apply business process analysis so that we make the systems as effective and as streamlined as possible before we automate them. And I'll talk a little bit more about that in a little later slide as well. I'm sorry, Sophos is wanting to do something and it would of course start popping up now. Also we want to take better use of expert systems and by expert systems I'm talking about building in logic into our systems that will take the expertise of our best attorneys, our best advocates in an area of the law and build that into the system so that we are able to take that knowledge and transfer it to these systems so that they are captured and used for people later on. I'll talk a little bit more about that when I'm talking about the document automation. And then finally we want to bring all of these systems together into a single point of entry in a state so that people can come in using natural language, assess their legal problem and then be pointed towards the most efficient method for them to resolve that problem, whether it be self-help, whether it be unbundled legal services or in some instances it might actually be full representation like our model that we have now. Business process analysis is something we want people to use at the very beginning. And essentially what it means is sit down and look at every step in the process that you have and then analyze whether or not this step is really necessary or if it could be streamlined in some way. This is an example of one of our programs with their intake methodology. And as you can see it started with a client talking to the intake specialist, the intake specialist getting the information, then the intake specialist would pass it on to the attorney. The attorney would look at it and say oh I need some more information, then that information here would go back to the intake specialist who then would get back to the client and then the client would get back to the intake specialist and the intake specialist would get back to the intake attorney. As you can see this could be very cumbersome and not the most efficient way to do it. They really hadn't realized that they were being so inefficient until they sat down and worked this out on whiteboards step by step and showed what was happening. And now they've gone to a process where the client gets in touch with the intake specialist. The intake specialist sends it to the attorney and if the attorney needs information they get directly back to the client. It makes a lot of sense but sometimes you don't recognize how inefficient you're being until you actually look at it very critically. So we want to be sure that before you start automating something that you come in and do business process analysis if you can because as I'm fond of saying if you take an inefficient process and you automate it all you'll have is an automated inefficient process. Now I talked to you about one of the categories being automated documents. In about 2005 LSE started funding an automated document server that is available to all legal aid programs around the country. Not just LSE funded programs but any non-profit legal aid program can use this. This was made possible by a donation of hot knocks of their product which is an automated document server that's used in many of the legal offices around the country and then also it was funded at the time by the State Justice Institute and LSE and there are now about 3,000 different forms up on this server that are available to people around the country and over 40 different states have forms up there and we delivered last year about 500,000 documents for people to use to represent themselves or for legal aid attorneys to use in their representation. Now the advantage of an automated document is that it's like a turbo tax type of form. It will ask people questions so that you can get through the process. We use another product called A2J Author that was developed by Cali and Chicago Kent and funded again by LSE and SJI and this also is available to free for any non-profit legal aid organization and to the courts to create these interviews but you can see that they're on a pathway to the courthouse and as they ask the different questions they will get the information they need to fill out the form. Now you'll remember that I said that it's oftentimes that the forms are not in plain English. They're not in plain language and so one of the advantages is it's not always possible to get the court to come back and rewrite the form but that doesn't mean that your interview that gathers the information for the people can't be in plain language and so this is one thing that we encourage people to use these systems for is because then you can ask very clearly what is your name what is the spouse's name using very plain language and you can also embed help into these so that you can have little question marks that can hover over and get help or you can even have embedded audio if you want so that it will actually or video links that will show people what they need to fill out the information. Now since we're very interested in plain language I'll mention to you now that the LSE TIG program our Technology Initiative Grant program has invested in several tools to improve plain language and you can find out more about those if you'd like at writeclearly.org. One of the tools is writeclearly.org itself which is a tool that you can use to analyze the language that you've used on a website or in a document and it integrates with a browser extension in the website or there's now an extension for it in Google Docs so that you can do not just the language on the web but also things that you're printing a document it will review it give you the grade level and give you suggestions on how to improve it. Another one that we have is called readclearly.org and this is a glossary of commonly used legal terms in English and Spanish and if you embed part of this code some small bit of code in your website then when one of the terms from the glossary appears on the web page the client or the user can hover over it and see a plain language definition of something like Exposed Factor or whatever the term is in plain language so they'll be able to better understand it and that again was writeclearly.org and the tools there are free for anybody to use. Now this is what the user sees you can see here it says please enter your full name and gender and then you also have down below hello Sally are there additional tenants or subtenants who rent the property you can very clearly ask the questions and get the information and this is all that the tenant I mean the user in this case is seeing they don't see everything that you needed to put together using the branching logic. Now you'll remember I talked about expert systems and so expert systems use branching logic to kind of build in if then else statements if you're familiar with programming you understand exactly what I mean but you don't have to show all of this to the user as they go through this but this is what it looks for the person that's built it you can see the decision tree here as you start through there where you ask a series of questions and depending on the answers it's which way they go with all of the branching logic so if they say yes then they go to one set of questions that they so say no they go to another and by using building in this expertise into this system you can avoid the user have the end user having to know all of this information and you can see that many times you can take a process like they did in Maine where there was a a food stamp qualification manual that is why I remember right was like a hundred pages long that somebody had to go through to fill out the forms to see if they qualified for food stamps but by the time it was reduced like this it was something like a six or seven page questionnaire electronic questionnaire that people could answer and it would tell them whether or not they were eligible for food stamps so if you do this you are very much able to help the user but build all this expertise into the system now you can also build this expertise into systems for the advocates themselves for example if you're going to use pro bono attorneys to help you and all legal aid programs have to spend twelve point five percent of their LLC funds on involving pro bono attorneys in the delivery of legal services you can build the same expert systems into document assembly programs for the pro bono attorneys so that they will be prompted when they're working with their pro bono clients so that they don't miss anything oftentimes a pro bono attorney might be a corporate council and so they're not as familiar with family law or housing law or many of the other areas of law they're quite common for the legal services clients and by having these systems with this knowledge they'll feel a lot more comfortable in taking on these clients we've got a project going on now in new york city called closing the gap where we're using pro bono attorneys and systems like this so they can help clients remotely they are able to reach out to them the client can fill out the information the pro bono attorney can then see those forms online before they get there and they can also if they want work with the client ask them the questions themselves as they go through the interview either way they prefer to do it but at the end they'll be sure they didn't miss anything because this was built by the experts to be sure that you ask every question in every type of substantive law area that they're assisting in so this is something that's very important to use and keep in mind as I said we really want to be sure that we build these things so that they're designed for mobile we want to be sure that we're using responsive design in fact we had a website study that was done under a grant that we got from the Ford Foundation and the e y intuitive that we worked with for this basically says that right now at this point in time we really should be doing mobile first type design so many people are using mobile devices that it makes sense to make the design to for so it will work with mobile first and then be translated to the desktop and so that's what we've been doing a 2 j author is what I've got an example of here and you can see that we've been working so that this way of filling out the forms is now available to them whether they've got a smartphone or a tablet or a larger tablet or if they want to get back over work on the desktop the latest study by Pew shows that 88% of the people in this country have access to the internet and that 77% of them have own a smartphone so this is a lot of market penetration right now and it's only going to increase this was as of 2016 so this is why it's so important for us to do that we have to be sure that we work with people in the most common method they have for many people this was their only access to the internet was that connection through their smartphone and so we have to be sure that what we build we're able to do it in such a way they can use it we are experimenting with adding voice recognition into these systems because sometimes it would be easier for them to actually speak their answers and voice recognition has gotten so much better in recent time you all are familiar with Amazon Echo or with Siri or with these types of systems and so what we really want to do is to start building more and more of that into these so that since they're using a smartphone it will be easier to use the voice recognition than to do the typing text messaging is something too that has a lot of advantages now as you can see here this is an example of the medical profession where they're actually using the smart phones to send text messages to people uh this is an example from a site called healthtap.com and here the client can or the patient in this case can actually text to the doctor and receive information back on their particular malady what it whatever it is that they need this for and so this is something that we should be doing more of in fact in one area that we might be doing more of is in the area of reminders this is a reminder i got a couple of weeks ago from my dentist when i had an appointment and so why aren't we doing more of this in our profession to remind people of the court dates that they have one of our programs in northern virginia started using this with their clients who were coming in to see them for appointments and they cut down the number of missed appointments by over 40 percent and so that's very very important when we don't have the resources to see and help everybody that needs us then we certainly don't want to waste those by having missed appointments so this is something that i hope that we're going to be doing more of is integrating text messaging we are tig has funded a court messaging system that will also be being done in northern virginia and the idea is we want to be able to remind people of their court dates and so if a person is going to court they oftentimes don't know where to go when to show up what to bring and so we can be sure that they are notified of this if we use this system and so we are experimenting with this it will have links to videos that they can watch how to information they'll be able to know what they need to bring to court what they need to how they can get to court what kind of clothes to wear this is something that's going to be very important for them and we think that we're going to show the advantages to the court just like we did to our program of cutting down on the number of contingencies that they have to have for people because people will be coming to court prepared and they won't have to say oh i didn't know i need to bring that can i come back later type of thing and so we think this has got a lot of it advantages for the clients and for the courts and we're hoping more and more of this gets built into it we know that there's a client portal very similar to this that they are building in in los angeles norio gato the chief technology officer there is building it to help them with their traffic tickets first to kind of walk people through the process so that they come online and all and then they hope to expand that into other areas as they see how it works another thing that i think that we'll start seeing more of our chatbots we're all very familiar with the chatbots now it's hard to go to a website where you start looking around for something and you don't have a little pop-up that says hi i'm julie can i help you find something well we all know that julie's not really julie jude's a little computer chatbot that is just out there to try to help us with what we need to know but why can't we be doing more of this in legal aid right now many of our statewide websites have the live chat feature embedded in them and right now we're depending on live people being available and they will see what the question is and then they'll be able to type out the answer well in developing the system we have developed canned responses for the most commonly asked questions well that's nothing but a human activated chatbot because what we'll be able to do is to take those most commonly asked questions that we found over the years for our live system and turn them into an automated chatbot so that when someone asks one of those questions or a variation that's very close to it the machine learning of the computer will be able to look at it and say oh that needs this canned response and that will mean that people will get assistance like this 24 7 and it means that the people that are actually doing the live chat now will be available for the questions that aren't commonly asked they can give individual tailor lot tailored tailored advice to the people that really need it i also think that some of this will be done over the smartphones as well ivy ashton who is the original founder of the legal server software did a demonstration i saw a couple of weeks ago where he plugged in his amazon dot to his computer and he had had it programmed to have a lexa answer commonly asked question about landlord and tenant law and i think we're going to see these types of tools made more and more available to our clients and i think we're going to see that this saves us a lot of time so that we have as i said more time to spend for individual attention for problems that can't be answered like this i think this type of technology will be introduced into our legal hotlines that we now run now and that more and more the answers that people will get about their legal questions even if they telephone us will be done by automated services such as this this is an example of a chatbot that's being used in the banking industry and you can see there are certain questions that you could ask and then it will give you information about your account i think that we're going to see this introduced for us to be able to use this might be something that the aba's free legal answers and if you're not familiar with that right now the aba has a program that they're trying to expand nationwide so that people can come online and anonymous anonymously ask a question and a pro bono attorney will be able to pick that question up out of a queue and actually then give a response to that question and i think that we're going to find that we can introduce something like this chatbot into an aba free legal answers environment and so that people will be able to ask their questions the machine the language will use machine learning to analyze that and see what the question is and then give the response that the person is looking for all automatically and this isn't pie in the sky other industries are using it and i think that's something that we're going to be seeing here as we expand our horizons with with legal services another thing this diagram this is a retail establishment that's represented in this graphic and you can see there's lots of different merchandise around right now we are using some low energy bluetooth devices so that people who walk around the store will be able to get information about that different merchandise these things are like this little blue thing on the wall so that when you walk by the merchandise and you've got the app on your phone you'll find out the specs on the merchandise and that is right now this pair uses 50 off i think we're going to start introducing these into places such as courthouses so that when somebody needs to be somewhere in the courthouse they'll be able to walk in and using an app they can tell it where they want to be and these little devices will be able to direct them to the right place whether it be at the clerk's office or for their correct hearing or wherever it's going to be i think we're going to be able to help people navigate if you think about it it's kind of like if you've been into the hospitals where they have the red yellow green stripes painted on the floor so that if you've got to go to the radiology department you know to follow the red stripe something like that i think we're going to start using low-energy bluetooth devices to give this type of information it could also be used at legal aid clinic so if somebody comes into the clinic that the app pops up and says have you already been pre-qualified and if you say yes and it says okay go over to this area and if you come in and say no i haven't been pre-qualified it says okay go over to that area and you go over to that area to start your qualifications which you then do on your smartphone because the bluetooth recognize that you came over there for that area and it would pop up the questions that you need to establish your eligibility those are just a few examples i can think of i'm sure that other people will think of even more promising ways to use these types of devices to help people navigate our system another way that we can do the take advantage of mobile is by videos and we are doing a lot with videos already but there's a lot more that we can do one of the reasons that we want to use videos is because they are so tailored for mobile devices as i mentioned more and more people their only access to the internet is through video and if you've ever tried to read a 15 page pdf that explains the divorce process to you on a pdf on your cell phone or your smartphone you'll know that that's really not a very pleasant experience and so we're going to see more and more videos for people because it's very easy as i said with a chat bot when they say how do i get a divorce in the state of georgia to send them a text message with a link to a video that they can watch off of youtube and learn all about how to do that and that all would be done without us spending any time working with them this is an example of some videos from the texas legal services center on how to navigate the texas court system and so that's exactly an example of what i was talking about there are some new types of videos that are being used called photo novella style and these have some real advantages it's hard to show you here on the slides but these never have moving video these are always just still pictures with the dialogue bubbles and they're up at the top and then there's an underlying audio that plays in the background that's saying the same thing that's in the dialogue bubble now the advantage of this is each one of these is its own individual slide and so that if the law changes you don't have to go in and reshoot the whole video like you might do if you use the talking head type of video you can come in and change just the single slide that has the information on the change in the law or the change in the court dates or whatever it is also if you want to have multiple languages you don't have to reshoot it by getting native speakers of those languages you can just change the the information in the dialogue and then have a different underlying audio track that runs with it so it's very easy to shoot these pictures and then change them to multiple languages one thing that we have to really concentrate on we do this is the design that focuses on the user it's very very important that we do that so often we as attorneys kind of think we know best of what the user needs and that's not always the case as you can see here here are a bunch of people that have come together and they are working to storyboard these out I'm sorry my slide is advancing automatically let me come back here before you even start writing code you want to sit down and you can see the post it notes they've got over on the right side of the screen you want to start working out all your steps it's like that business process analysis that I talked about and then you want to bring users in to help you and walk through those steps and say oh this might work better if we did this or this might be work better if we did that do that before you build the system so many times what we do is we come in and we get all the experts together and we build the system and then we bring the users in to test it and that makes it much more expensive to change it because we might have it completely wrong and have to start over from scratch it'd be much better to storyboard it out do the user testing beforehand then start writing the code and then bring the users back after they've had some initial input to this this is an example of a really clean design that was redone for Illinois legal aid online after they did a lot of extensive user testing with their older design and they found out some very interesting things as they went through this for example they were going to use one of the categories that said my business and then they asked people that okay where would you put child support and one of the people said oh if I was looking for child support I'd look under my business I said well that's interesting why is that because my child support is my business and so our assumptions about what people think things should belong in is not always the same as the actual people were trying to reach and so they've tried to redesign this into areas that don't say you know housing law or consumer law or something like that instead of consumer they have money and debt the way people actually think about their problems as opposed to the way lawyers think about their problems here's another example that we're doing some very targeted type of user design we have stateside legal which is a project that's run by pine tree legal services in Maine to have information for the veterans and their families and we are having a very targeted section of this now for women with military service because we're finding some very unique problems that the women in the military face after they've come out as well they're very different from just the general problems of the men and so we're not going to try to make a one size fits all by using this type of user experience is very important that we target particular users we have another site called disaster legal aid that is designed for people who have suffered a disaster like it started with Katrina the hurricane in Louisiana it now can be used for people that are subject to flooding or for wildfires that need to know how to get in touch with FEMA and so it was part of the recovery after the sandy disaster we're finding that the user experience is very important and so we don't have to build just one website that serves everything we can have niche websites that serve a particular clientele my law BC is a way that they're building the expert systems into a website to help this this is in British Columbia this is one thing that as you get more into access to justice you'll come to understand this is not a problem in the U.S. only that this is a worldwide problem of providing access to justice and we need to be looking at how other nations are solving these problems as well the technology initially grant program that I work with here at LSE has a conference each year in January and we have people who come from Australia from South Africa from England from Canada every year to see and exchange ideas with us on what they're being doing my law BC starts out with the same kind of guided questions that you have here so that you can answer a few questions and they can tailor the responses that you're going to get to your particular situation and so this is something we're going to start building in instead of just taking people to a website and having a bunch of different areas that they can look in we'll start asking them questions that will help us find out what information they need and I'll have a little bit more about that later when I talk to you about the actual portal also as I talked to you we're looking at some targeted areas this is something called families change this is a site from the California courts but they got this from a British Columbia site that they liked and they worked with them so that they could tailor it to the United States and then we have a tick grant to help one of our other programs in the eastern part of the country to take the same site and target it for them one of the things that it uses is gamification excuse me we have to experiment with new ways to reach things and so they build a game called changefield that will help children kind of understand what they can expect as their parents go through a divorce and help them you know it's not necessarily going to make it easier for them it's going to be a very trying time when for children when their parents go through a divorce but at least it will let them know that they're not alone in this that they can start learning about the process and it will give you know give them help as I said coping with their feelings so that we can provide very tailored assistance to people in these types of situations using gamification if you're interested you can go out to changeville and play with the games and see what you think about it another one that we're doing is called represent this is a project that we've been doing in Connecticut and it we've been working with a university that is very much into using technology and design approach in providing assistance to people and they started this as you can see you will choose your thing are you ready yes your honor thank you sure dude so what would your response be yes your honor thank you or sure dude and so it can kind of give you I mean that may seem obvious but we've seen people come into courts in t-shirts and flip-flops and things that probably don't set the right impression that they want to have when they're appearing in court trying to get custody or something like that so this will walk them through different scenarios and let them learn from this about what the best approach is when they're representing themselves and presenting the court case another project that we are seeing from overseas is called wreck visor this is a project in the Netherlands this is a project to use online dispute resolution to help people resolve their divorce situation one of the things that i liked about the wreck visor project and it's being replicated in british columbia unfortunately the actual dutch project itself has been shut down because of some funding problems but the idea still is applicable to some of the things we do here and to the british columbia project because they had essentially three stages in there one is the odr that the people could come in and using the online dispute resolution could try to resolve their own issues and if they couldn't they could ask for some coaching to help them get that through that with a mediator and if the work with the mediator didn't work then they could ask for an adjudicator who would come in look at both sides and make a decision of what they thought should be in the separation agreement but then at the very end of the agreement they had a third party who did an analysis to look at what the parties had agreed to to see if it looked fair to both sides and if it didn't then they could come back to whichever side look at mediations they might not need to make and make sure that this was informed consent to make sure that this was something that they were agreeing to not because they were coerced into it by the other side but because they thought it was fair and so we're going to see more and more of these types of things the legal services corporation is funding a project in orange county to introduce odrs into small claims cases uh there is some work underway in new york to do this in consumer cases and so we're going to see how online dispute resolution can be introduced into these different projects now as i said we bring all of this together into a single point of entry for people right now the legal services corporation pro bono net and microsoft are working on building a prototype of the litigant portal the litigant portal is a way like i said that we can bring all people together into one single point of entry to get guided assistance with their problem you ask well why is that important well currently as jim sandman our president and all the c said the current system of us accessing legal services of confusing opaque and inefficient for me many people excuse me a second the reason for that is right now we have legal aid programs multiple ones in each state we have the courts they have their own websites we have the bar association it may have a website it'll have legal referrals so there are multiple providers with multiple websites and multiple procedures to get to the same point resolving their legal problem they don't know which one is the best place to go they don't know which website to trust they don't know what's relevant to the case what's not relevant to the case it can be very confusing and we have to remember it's going to be difficult for us to build systems that can turn lay people into lawyers to represent themselves when it took us three years of law school to get to that same point and so we are partnering i said with uh pro bono net and the microsoft and we're working with a lot of people such as the a b a the national legal aid and defenders association the self-represented litigation network with the state justice institute and we have gotten two pilot jurisdictions alaska and hawaii who are going to work with us to stand these up in their states so that we can really learn how this is going to work in practice and we are envisioning a way that is very personalized to the user you remember i told you earlier that we have to consider the user first as we develop these things and so that's what we want to do we want to do something so that when somebody has a problem they can come in and type in that problem using natural language that the portal will be able to use machine learning to look at this and assess how this problem uh what this problem really means even if they don't express it in a way that people normally would that the results that we return to them are tailored for this individual user so we're not just giving them 2000 different results about divorce in this particular state but we're giving one that really relevant to their problem and it gives them next steps on how they can get through this process and so like i said we want to be sure that we consider mobile first this is something that's very important it's going to be in a single platform that comes in we've got to make sure that any type of technology they have it's accessible that it's got to be tailored to their individual solution it's not one size fits all that gives them guided assistance so that step by step they're taking how to resolve their legal problem so that they would resolve it and it's got to be integrated into the other systems in the the jurisdiction so that when they finish this the results that they see may have information from legal aid it may have information from the bar it may have information from the court it doesn't matter where the good information is the best information for them to use to resolve their problem is going to be brought together in one place and they're going to be given as i said step by step instructions of how to use this assistance to resolve their problem now on the left you'll see that there is a this is the kind of multiple provider problem that i spoke to you about the state courts libraries professional directories self-help centers legal aid organizations the list goes on and on of places that provide assistance to them the portal is where we bring all that together in the different modules and it may be the problem identification module to help them find out what it is it may be the solutions options that show them all the different choices they have whether it be self-help unbundled legal services or full representation there's going to be the portal administration for the people the providers come in and they actually curate the experience for the user so they can say these are the trusted sites that we want people to see they don't have to look at all thousand sites around the country that had this information these are the sites they'll get and we're going to start out in four areas to test this housing veterans affairs family law and consumer and so that's where we're going to start and then the outputs they're going to get the completed documents they might need forms they might need and we're going to get back usage data service rankings portal feedback because we want to be sure that we get information back from the providers and back from the users so the system gets smarter and smarter we want to be sure that our referrals get better and better this isn't something that we want to do and then just hand people off and never find out what happens to them it's very important for us that we be doing you know that we be doing the right referrals to let them actually resolve their problem and again we really want to focus on the user design for this that's very important Microsoft is committed that we're going to use open standards so that this portal will be something that can be used with multiple different systems and we want to be sure that we use inclusive design so the people with disabilities will be able to do this we want to be sure that natural language processing is built into it so people don't have to know legal jargon to describe their situation and we want to see this being incorporated into the systems in a jurisdiction so that as I said it gets smarter it's more comprehensive and people can trust that this is one place where they can come to actually get the help that they're going to need to help them resolve their problem and Microsoft is committed to spend at least a million dollars on this I think they'll probably spend probably two or three million dollars on the project they've got a cracker jack team of engineers on this pro bono net is putting resources in here to help us with project management and facilitation and so I really have high hopes that we'll be able to bring all those systems that I talked about earlier in the presentation together in this portal so that we can help people help themselves to resolve their problem and start really working to achieve 100 access to justice I told you early in the presentation that I would give you links to the summit report there's also a report here from the national center for state courts that talks more about how you can build a litigant portal so I've been talking nonstop now for 50 minutes or 51 minutes and 14 seconds actually and so I think we will stop here and see if anybody has any questions hey Glenn I want to say thank you I guess I'll start off with the first question which is I know you mentioned in terms of the hat docs and a2j author that it was used over 500 thousand times a year is there any data on the back end or any analytics or evaluation tools that can measure how you know what is the result of the use of the those programs so does a person get their issue resolved um yeah well one of the problems Miguel is that when it leaves our once it's assembled by the user we don't know what happens to the actual documents so we funded a study in michigan to actually look at the document angela trip in michigan developed a really good system there for divorces and they sent out a bunch of law students and others that they hired to go look and actually pull the physical files of I believe it's over 6000 divorces and that study is available and what that study showed was the people using the automated documents from law help interactive at a2j in michigan that they were able to finish their divorces uh actually in a little bit less time than those people who had had lawyers and so they tried to look not just at the simple uncontested but you know all the different categories and so uh that study is available to look at but what we saw from that I know it's just one study but it was very encouraging that showed those people had very good results and were able to finish their divorces and were able to get you know results that were comparable even with having an attorney oftentimes quicker than those people who used an attorney that's great thank you yeah Miguel hit on a really important point there which is um the evaluation and the feedback loop that is needed to do um iterative design and try these projects um Glenn what are some of the other best practices you've seen out there for trying to uh get feedback on projects well right now we're working with the national center for state court they are standing out by projects on Oasis which is a national standards group they are the ones that have developed the ECM-4 standard for e-filing and Tom Clark will be and I'll be working with that group as well to develop some data points that we can transfer back and forth so what we are our plan is to do is in the first iteration of this is basically make referrals from the portal to the end providers but we want the uh the version two and version three to start getting information back from those providers as well so as we integrate like in Alaska they're going to be doing e-filing start and when they do the e-filing we want to be able to pass the document over to the e-filing system but then we want to be able to track that with a unique identifier so that as it moves through the court we can see does it get stalled does it get finished and that information automatically will come back to the system so that the administrator the system will be able to go look and see oh they e-filed this document and this many of the people that did that got their divorce or that this many people that did it you know forestalled their eviction or whatever the e-filing process is that they were doing also we want to provide it to the providers when we give a referral let's say that we do all of this we refer this to the local legal aid program we want the local legal aid program again with a unique identifier that's not where we can't identify the client but we can identify the referral to let us know whether or not this was a good referral in other words did this person really qualify was this something within their priorities that they could actually take and you know was this something where they actually were able to help the person so that we know if we have another person with those same characteristics that we should either send them over there or if it wasn't a good referral somebody else with those same characteristics we won't send them over there and so that's something that we're really trying to improve with this portal are those outputs to see whether or not they were actually helped right now as I said with law help interactive it's very difficult to do this because we don't get any feedback from the users now we're also experimenting with doing text messaging to the end users of these the different solutions so that they can very quickly reply to us was this information helpful to you in resolving your problem now that's not as good as actually finding out whether they got through the court hearing and successfully or not but still just knowing from the end user using the text messaging this is something we think that we can start tweaking and improving our our system and then also we're trying to do this incorporating into the website of was the information we provided you on the website useful so we're building more and more client feedback into the systems so that we can refine and improve them but this is something the Pew Territable Trust is very interested in we're working with them as well they've been have they have a group of experts working on white papers on this to help us with the portal design so that we can do this because as we start trying to solve or provide 100% access we need to know which methodologies work the best so that as with limited resources we focus resources in those areas so would like to remind people at this point that you can either ask questions through the question box or use the raised hand option and we're happy to unmute people so that they can ask questions Glenn really has some very unique knowledge in that he sees all of the different projects that are going out on out there if there's a particular project that or idea that you are interested in it'd be a good idea to ask him and he may know other organizations that have tried something similar or best practices or things that can be pulled out of those past projects okay we've got one here which is you mentioned that providing services that avoid the use of legal jargon is there an approach for language barriers for individuals who don't speak or understand English very well yes we have quite a few projects that are very interested in language access as I mentioned to you we are trying to design these systems like the photo novellas so that we can have the same videos being in multiple languages we are going to to actually design the portal initially just in English until we actually do the pilots but then it's going to be very important to us to provide other languages also our system of statewide websites if you start looking at those you'll see that many of the sites have complete mirror sites in other languages especially in Spanish we're also experimenting with the machine translation to see if this is effective enough we've seen a lot of improvement from Google in the last few months just in the machine translation what we're trying to look at is to see if we can do a combination of that so that we start the translation with machine translation but then we have the certified interpreters come in and do the check it double check it just to make sure that we don't have anything that's gone wrong so a language access is something that's very important to us but one of the things that we tell everybody is before you translate this into other languages translated into plain English first because it's very important in any language that it be taken out of the jargon and legalese and into plain language and it's a lot less expensive if you do it first in English and then translate from the plain language English into the other languages oh also there was a project that we've done at SART's own program Northwest Justice where we've used the sign language the American sign language for intake systems using video and so this is something we're very conscious of too is that's going to be something like I said that as we can introduce video into this that we're hoping to expand by using ASL right I dropped a link into the chat which has a link to Northwest Justice Project's YouTube channel it has the photo novella style of YouTube videos which are very easy to update as the law changes because you don't need new actors you just need to shoot a very small part and also change to multiple languages the ASL videos are also there we've got another question here which is my question is with regards to the digital divide I understand you spoke of emphasis being placed on projects that are mobile centric but was wondering of other ways to communicate with communities that are attempting to address that I like working with libraries and schools in order to provide access to the technology itself I'll let Glenn answer then I've got a little bit of a follow-up on some broadband stuff that Northwest Justice Project has worked on thanks in 2010 as a pre-conference to the LSE's TIG conference we had a special program for librarians to educate them on the resources that we have available through our statewide websites and legal aid programs and so we've been expanding more and more of the actual partnerships with the libraries and then just last year we got a grant from Mellon Foundation to investigate doing a curriculum to help public librarians with the training on these particular resources that was completed and now LSE is looking for funding to actually build that curriculum so this is something that's very very important to us is the partnerships with the libraries just two weeks ago I gave a presentation on access to justice for the Maryland and Delaware library associations and I've done presentations also for the law library association at one of their national conventions and the public library convention as well so this is something as you point out it's very very important for us to work with the libraries on this and in fact we have actually on the pro bono net template which is one of the statewide website templates the law help template we have worked it out so that if a library wants to partner with us they can actually brand a site that looks like it's the site of the individual library but it will return the legal information content from the statewide website in that jurisdiction we've done partnerships like this in Kentucky and or I can't remember Kentucky or Tennessee and in Minnesota and in Florida so you're correct this is something very important and something that we are hoping to have this curriculum done this during this current year so that libraries will be better able to direct people to trusted resources like those of legal services if the part of the funding for Northwest Justice projects YouTube video launching came from a BTOP broadband grant that we worked with partners including librarians and schools and community centers to go after that funding there is definitely a large push in the Northwest community to look at issues of net neutrality and lifeline broadband access for phones and how those issues affect individuals that are target audiences for our clients so that type of partnership is essential to getting the technology and the literacy in the hands of individuals to be able to access these online tools well and one other thing that we're doing too is encouraging the access to justice commissions which most states now have one to reach out to their library community and not just make them aware of what's going on get them on the commissions because one of the things that libraries can play a very important role is as they see their patrons come in and look for information if that information isn't there they can report that to the access to justice members to let them know hey lots of people need help in this area and there's not a resource for us to send them to and that's one way to really find the holes in your system is to get that kind of feedback from the libraries yeah that is definitely a key point for doing these types of projects if you can try to diversify the stakeholder base so that you have community partners librarians educational institutions and professors that care about these issues your chances of adoption and finding innovative solutions go way up in law we tend to stack committees with a bunch of lawyers and that interdisciplinary approach adds a lot more opportunity for innovation and seeing things that we wouldn't normally see okay i guess see it see it as there's no more questions i want to thank you again glenn for taking the time to present this afternoon and we really appreciate you if possible would it be okay to share your powerpoint and your email address that the fellows wanted to follow up with you over the summer absolutely that's why i put it in there oh thank you perfect well and i just want to say how happy i am that you are participating in this program and i think that you're going to be a great resource for the legal aid programs and i'm here to help in any way i can so thanks for the opportunity to give you kind of this overview