 My name is Rich Leonard. I'm a judge with the bankruptcy court in North Carolina and here today in my capacity of an all-star team-fed team, this is my fourth privacy conference and my ties here are both professional and personal that Dean Douglas who welcomed you yesterday actually started his legal career in our spare bedroom in Raleigh waiting on his first paycheck from my wife's law firm so he could rent an apartment. So we've come full circle. We have a lot to talk about. Our public access program or PACER it is is always known has hit some remarkable benchmarks in the past year because we finally do have all federal courts including the appellate courts using the same case management system. It is now a universal database of cases in the federal courts. We have just signed up our one millionth account holder and are adding about 5,000 new account holders a week and we're nearing the end of a massive assessment survey of all of our users that's giving us valuable information about where we need to go in the future. We have some new developments that we're proud of and we hope to talk about and display for you today. I think no one can really dispute that PACER is the most robust and comprehensive access program in any court system in the world currently and those of us who have had a hand in it over the years are proud that we originated it entirely within the federal courts without any external requirements showing what remains our complete internal commitment to transparency and accessibility. But at the same time our public access efforts are under attack in the media and the blog spheres as never before. We are charged both with not making our information sufficiently accessible through commercial search engines and somewhat paradoxically at the same time, faulted for not protecting the privacy of the litigants whose interests are behind our firewalls. Their objections to the fee based system that Congress has instructed us to implement and accusations that we've distorted the concept of public access to spend fees on extraneous projects. In my view at times the rhetoric has outrun the facts. We're here today to tell you where we are to hear from you and to engage on any issue on which you would like to engage us. This is how we're going to proceed. First, Judge Haynes and Ms. Del Monte are going to bring us up to date on the continuing development and tweaking of privacy rules for litigant information in the federal courts. Second, Judge Smith and Ms. Solomon are going to wade into the surprisingly tricky waters of opinions exactly and particularly at the trial court level. What are they? How do we classify and distribute them in neat ways that they can be enhanced with new technologies? Next, Judge Haynes and Mr. Schedule are going to talk about and demonstrate a new product of which I'm particularly proud that we will soon offer through Pacer where we actually allow you to access the raw digital recordings of court proceedings. Next, Mr. Schedule is going to show you the new search engine that Pacer has developed that we're about to unveil in the next few weeks. Next, Mr. Schachin is going to walk you through the comprehensive user assessment that we have just completed that shows us pretty clearly what are perceived to be by our actual users, our strengths and our deficiencies. And finally, Mr. Schachin and I are going to talk briefly about the massive effort already underway in the federal courts under the acronym of next generation or next gen as we attempt proactively to build our case management system for the future and thus derivatively our public access system. And then we hope to hear from you. It's a lot to cover. We are confident we can do it and have time to engage as much as possible with each of you at the conclusion. So without further ado, Judge Haynes, Susan. Thank you, Rich. My name is Jim Haynes. I'm a bankruptcy judge in Portland, Maine. And Susan Del Monte is a from the administrative office of U.S. courts. She's a staffs the KACM committee, which is the court administration case management committee of the judicial conference. We've worked together on public access and privacy issues for some years now following the work that was done by Rich Leonard who preceded me on KACM and really was there at the infancy of the transition to electronic public access from the old practical obscurity days we talked about yesterday. Excuse me, Judge Haynes? Sure. And there we go. So it's, and I think it is important as Rich mentioned that the existing public access mechanism is available broadly and we'll talk about the dimensions of that. Many of you are aware of it, but it is derivative of the case management system, the electronic case management system. Some time ago, back in 1999, really the federal courts were on the threshold, some of the bankruptcy courts across the threshold for electronic case management, managing their files electronically. And some bright lights ascertained that if we have a digitized court record system and case management system, it can be made available to the public and provide the kind of access at desktop computers that could otherwise previously only had by going to the courthouse and looking through the files. But in anticipating those developments, the story that I have is that at one of the KACM meetings they took a tour of a bankruptcy court and thumbed through the bankruptcy files for a typical consumer bankruptcy. And we're really, those of us in the bankruptcy field, see this stuff day in and day out. And so I wouldn't say our eyes glaze over on, but it's the sort of stuff that we expect to see in bankruptcy files. But district court judges and appellate court judges took a look at it. And in conjunction with the bankruptcy court judges went, wow, do we really want all this stuff about, you know, the children and how many teddy bears there are in the bedrooms and all those things. We really want all that just out with no filter available on the Internet. And so starting in 1999 with the KACM subcommittee started considering the implications of making all these filings available on the Internet. And developed a policy that was adopted by the Judicial Conference, which is the governing policy setting body of the federal courts in 2001. That policy was initially aimed at providing access to civil case files over the Internet. And it was later after further discussion and study, we decided that the Judicial Conference decided that access to criminal case files could be had over the Internet as well. There are different concerns in each, and we'll talk more specifically about some of the ongoing issues that result from criminal case files in particular. But as of 2003, subject to the judge's traditional ability to seal cases or pleadings, civil case files and criminal case files were all available subject to this Judicial Conference policy. The Social Security Disability Benefits cases, which typically have a lot of medical information, were not provided via the Internet for public access. Again, a nod to the privacy interests of the individuals whose cases were being litigated in the courts. The docket and the opinions would be available, the orders, but the party filings would not be out on the Internet. Just as in days before, those would be available at the courthouse if somebody came into the courthouse and asked to see the file. They could see it there, but it wouldn't be immediately broadcast and available. Those inner parts of the Social Security files wouldn't be available on the Internet. In connection with the making available of civil case files on the Internet, again, looking towards privacy concerns, the policy required that personal identifiers be redacted. Again, because court files reflect what the parties put into them, the redaction responsibility was placed on those who file. If you want something protected, then you take the responsibility to protect it at the threshold after it's filed. The court file is what is filed in the court, but Social Security numbers were limited to the last four digits, dates of birth, only the year, financial account numbers, the last four digits, names of minors, just initials, and no home addresses, only city and state and criminal trials. There were exceptions for voluminous case files that basically involved the transfer of the content of files created in another forum into the federal courts. For example, a case that was removed, we would not say that the redaction would have to basically be post-fact and go back and take place to what came in from another court. Now, the policy was designed to work in connection with Pacers. So, Pacer is the vehicle by which parties file cases, file pleadings, and through which the court distributes opinions and orders, scheduling matters and all the like. It's the workings of the cases available on the Internet, but that availability is not restricted to participants in the trials. Anyone can register through Pacer, get a Pacer account, and obtain access to the materials. And I guess the key issue there, and some would take issue with it, and some wouldn't, is that in order to get the access, one has to go through the Pacer portal and register and be responsible for paying the cost of the files. But at the same time, it provides a tag so that one can ascertain who's been into the files in the event that there is a misuse of information from the files. The judiciary doesn't allow, as I said, anonymous access to the files. But via Pacer access has really taken a lot of burden off of clerk's office, so in the cases of big, important, high-profile files would be inundated with people at the desk wanting to go through the file, and many of whom would never get to the head of the line, so they get access to the information, less burden on the clerk's office. It's available to people who participate through Pacer registration. Now, ultimately, and the reason this came out as judicial conference policy is because, as most of you probably know, the federal rules promulgation process is lengthy. It takes years to get a federal rule as a federal rule, and in 2007, actual federal rules were created that mirrored and pretty much promulgated, again, the same judicial conference policies with a few exceptions. They added a waiver provision, which really makes express that which was implicit before. If you file something and don't redact it, then you've waived your concern about it being available. It created public access, it created restrictions on immigration case filings by the parties. They were treated like social security cases, and included an express list of items that were exempt from redaction requirements. Now, the Civil Rules Committee basically looks at when the rules are going to change and whether there's a need for the change, the CACM Committee basically looks at policy decisions relating to the accessibility of public case files, among other things, and very often, the CACM Committee does studies through the Federal Judicial Center to ascertain what the import and impact of pilot projects relating to public access is. We did some of that with regard to the digital audio program that we're going to talk about here in a minute. But at the behest of the Rules Committee, since the rules have been in place, we have conferred with them, and they've conducted some surveys with judges and attorneys to ascertain how well these rules are working now and whether or not there is a need for further amendment of the federal rules to either expand the redaction requirements, restrict them, change the categories of cases that are subject to redaction and subject to pacer availability. And as of the last joint meeting on privacy between the CACM and all rules subcommittee, there really is not a demonstrated need yet to do something more in the way of rules amendment and promulgation. But there are concerns, and some of these concerns are going to take some time to figure out exactly how to deal with them. It's been, based on the surveys that we've taken with the courts and attorneys and the feedback we're getting and our discussions with the Rules Committee, there are a lot of things that are of import and need to be addressed, but are best left to development at the local level in light of circumstances within different districts to determine how best to handle them. One good example is cooperation provisions and plea agreements in criminal cases. There is concern that there are those who would gain access to the content of a plea agreement that includes cooperation provisions and use that information against the cooperators and thereby disrupt and intimidate criminal trials. We don't yet have any, there was a high-profile discussion for a while of a website called who's a rat dot com, which purported to identify those who had turned states evidence or government evidence in criminal trials and sort of single them out for identification and potential intimidation. But there really are, as yet we have no reports of actual harm coming to a cooperating defendant or witness or others participating in a criminal trial as a result of pacer access to criminal documents. And that's not to say that it's impossible that it could happen or that it hasn't happened, but there are so many sources including the exercise yard in the prison for that information to get out and around that it's just is impossible to tell but really there's nothing and again we can tag this access to the files through the pacer system that shows that there has been any harm resulting as a result of the availability of criminal case files. Now there are a lot of different approaches to how to deal with sentencing with plea agreements which include cooperation and wrap this up quickly, but some courts don't make them part of the criminal file. Some of them break the cooperation part off of the plea agreement. Some courts put in a dummy docket entry in every case that could be seen as plea agreement and it's either going to be blank or sealed or like there's a number of steps that can be taken. One of the things that we're working on and have created is a range of options of accessibility for files, for items within court files which would enable the court not only to just absolutely seal a pleading or a document in a file but to make it available, judge and attorneys only, court staff only, one party in the case of an ex-party matter and the like which again trying to give the judges tools to manage and protect privacy while not sealing up cases completely and just saying nobody can see anything that's gone in with it. So we haven't really decided on what the best practices are in with regard to those those problems, continuing to study it, also continuing issues relating to whether alien numbers should be redacted or not. They're like driver's license numbers for aliens and they come up in the files where aliens are being prosecuted is probably the most likely thing and courts of appeals very often use those really as identifiers for those defendants. Social security numbers, we've heard about their sensitivity and how they may be able to be replicated from the last four digits if you know where someone was born in the year. There's a proposal that's under study right now in the bankruptcy area to provide a different unique identifier, not the social security number or to somehow manage those identifying numbers in a way that is less risky but nonetheless that's under study and as I also indicated we're talking about different levels of access and one thing I wanted to ask Susan about was we have rules and we don't have expressed sanctions and I think we saw some provisions yesterday where there were expressed sanctions for not redacting but the our federal rules don't include expressed provisions for sanctions because they are the rules of procedure and the judge has inherent authority to enforce them and see that they are they're observed and I think we've come up with policy and we've implemented the policy and now the issue is enforcing the policy because as we say our policy does put the onus on the people filling the court file to take care with what they fill it with and then our obligation is to make the court file available as much as possible but Susan you've you've seen some instances where there has been enforcement. Yes there are not a lot of cases out there interpreting the federal rules the privacy rules but there are a few. Generally when someone finds a violation of the rule a social security number that wasn't redacted or some other information that a filer forgot to redact there are a bunch of options that the judges take sometimes altogether sometimes just one or two at a time but generally we see some sort of a reminder to the attorneys about the rules and to redact in the future. Secondly they typically remove the document from internet public access sometimes they seal it sometimes they just make it available at the clerk's office but not on pacer but they do limit that internet access and when they do that there's also a requirement that the attorney refile a properly redacted document usually within a very short time frame as little as 12 hours sometimes 72 hours so they give them a window to get a properly redacted document in the file so there is still public access through the internet to something. In a few rare cases there have also been other sanctions against the attorney who failed to redact. We heard from our men our Minnesota panel yesterday about the decision of the chief judge in the district court there that's probably the largest sanction at least that I've found in looking at the cases. That was credit monitoring for over a hundred people whose social security numbers were accidentally disclosed. Interesting thing in that case the attorney who made that filing was the one who brought the issue to the court to say wow in our attachment in this case file we included this information that wasn't redacted and we should have. So the law firm actually suggested the credit monitoring for a year and the judge agreed with that and then also put in a $5,000 donation to a food shelf so that they'll maybe think twice about this next time. There's been one other sanction this one did not involve money but I thought it was sort of an interesting remedy in the eastern district of New York last July. A judge denied a summary judgment motion without prejudice to refile but denied the motion because of a failure to redact as required by rule 5.2. They did it in a text order on the docket sheet which is something that we're able to do with our CMECF and PACER systems but it's kind of an unusual case. Then there are a couple cases that are pre well there are three cases two of which are pre-rules so they were under the old privacy policy where someone moved for sanctions under rule 11 someone whose information I think in most cases it was the defendant who made the filing that included the social security number the plaintiff moved for sanctions under rule 11. In one of those cases it has survived to be examined not under rule 11 but under a civil contempt sanction and that's not resolved yet but in the other two the judges found that the parties hadn't met the safe harbor provisions of rule 11 and so they denied the sanction motion in all of those though I think it's important to note that once the redaction error was pointed out to the courts those documents were immediately removed from internet access again with the direction to refile the properly redacted document. I'll just say in closing that we're not self-satisfied with the policy it's continuingly being re-examined both by CACM and we just had this recent interaction with the rules committee we're going to have a forum at Fordham Law School in a couple weeks to take even more comment about the current state of privacy and public access in the federal courts and just every time you touch one part of it there's a ripple somewhere else and we're aware of that and probably the most one of the most striking examples of that is with regard to transcripts and I'm not going to spend a lot of time on that but court reporters make money from selling copies of transcripts and but at the same time when the transcripts have been filed they're part of the court record and there was a very delicate extended negotiation that went on for a long time before we hit our current transcript policy which says you order the transcript when you get it you've got so much time to go through it and redact again that's the party who requested and will be filing the transcripts responsibility but then there's a period of time after before it's fully available to the public during which people who want a copy of it have to order it from the from the court reporter not so much a problem with the digital audio recording but there are still a lot of court reporters out there and we'd have to balance you know their issues although what we were trying to do was just get public access to court records and find out that there were other dynamics involved that had to be wrestled with. Thank you. Thank you Jim. Thank you Susan. We're going to move now to our first circuit team with Susan Sullivan and Judge Will Smith from Rhode Island to talk about opinions and we're gonna have to move around as we do these presentations to run them from the center laptop which means our name tags will be totally redundant shortly so. Right here. Good morning. I'm Susan Sullivan. I'm the circuit librarian for the first circuit US courts. That includes both the Court of Appeals, the district and bankruptcy courts within the first circuit. I'm not going to ask everyone to tell me what states are in the first circuit but we do cover Maine and New Hampshire, Massachusetts, Rhode Island and Puerto Rico. Judge Smith and I are going to be talking about opinions and my portion of this presentation and I will I would really like to have Judge Smith speak the most is to just give us a little bit of a review of all of the thousands of documents that are in Pacer and are really generally known as court records. There are those documents that we call written opinions or again those documents that are going to be used and available to the public, to lawyers, to attorneys, any number of individuals out there on the web to do legal research. So I'm coming from a little bit of different perspective. I'm coming from certainly this perspective of helping individuals find opinions. So with first we have to talk a little bit about generally how many decisions are issued by federal judges per year and then how many of these are considered a written opinion of the court. I don't have a number. I didn't pull out a number and you will find various numbers of certainly how many decisions are issued and then particularly how many written opinions but this is the definition that has been approved by the Judicial Conference of the United States and it says that a written opinion is defined as any document issued by a judge or judges of the court sitting in that capacity that sets forth a reasoned explanation for a court's decision. Pretty good definition at least in my personal opinion but perhaps the word debate is a little harsh word to use but there is a difference in interpretation of them and I think two of the clarifications that came with that approval of the definition is the responsibility for determining which documents meet this definition rests with the authoring judge and certain judges are going to look at documents. Some judges may hold back and not tag certain documents as written opinions whereas other judges may actually tag as a written opinion almost every document that they issue and certainly the decision as to whether a document meets this definition is not the same. This is another clarifying point about whether an opinion is to be published and I really am not going to go into published versus unpublished and get off on on that whole debate but that's also something that takes place at the time that the judge is deciding whether it is going to be tagged as a written opinion. If anyone has any on the panel has any comments about the definition I would be happy for them to chime in. I certainly was not in on the writing of the definition. Actually I was in on the writing of the definition a little bit and I don't do that many written opinions because one thing that's it's really gotten me to rule from the bench and now with the digital audio availability that's not really cover either and also reasoned and I go well if I say it's reasoned yeah I guess I have to publish it but if everybody else finds it completely unreasonable you know I actually had a big hand in the first crafting of this definition and I thought it was workable but the empirical evidence would be to the contrary because when you run searches against the databases of various courts you find similar size courts that will have three opinions in a year versus 14,000 so obviously we're not all on the same slate at this point and I've actually suggested some language that might push the ball a little further up the road but but we'll see I think we ought to tie it more to opinions than in our view would have precedential value for a useful value for subsequent cases not just because that case itself appears to be reasoned but but we'll see and I'm just to take the other side of this if we're talking about transparency and these opinions are available at no charge because they are the court's disposition of a matter then I don't think they need to be a presidential matter people should be able to see what the everyday garden variety mill run case reasoning is and take the other side what you've done is reinvented practical obscurity because you create an opinion database that has so much trivia in it that you can't find anything useful so yeah Jim and our good friends and we frequently don't see it the same way the friend part or the opinion part I think that the reason for the disparity is a lack of consistent protocol in chambers and a judge's inattention to the judicial conference policy that requires him or her to take a look and either designate something as opinion or say that's really just an order but I and I think that if they took that step within each chambers you'd find those statistics would equalize but rather than just inattention not for the German rich yeah something to think about so anyway this is just some types of documents that if you actually have looked at a written opinion report in Pacer you will find sort of these descriptive words there there really is a variety if you look across the across the courts this is just a diagram or a chart that explains that the judge or the panel if it's a court of appeals designate something as a written opinion it is then loaded and they work through CM ECF it then appears in Pacer the availability then of these documents that have been designated as written opinion which again our orders memoranda decisions and so forth they're available through Pacer they are definitely available for the individual from the individual court websites and then obviously they're also available through the fee services or certain free websites that also gather the federal court opinions this is just an example of a district court website the district of Idaho Judge Smith chose Idaho and if you'll notice if you're at the site you do find where it says written decisions now even this terminology of written decisions will be different at different websites some will have a link or a button that says opinions some say written decisions it there is a little bit of a confusion when you're at a website for the first time but obviously the more you use that particular jurisdiction you'll know exactly where to go then they're the district court written decisions there's also bankruptcy written decisions you have a choice of you of how you get to those decisions I'm going to very briefly mentioned court web which I don't know that everyone knows about but court web was developed in the Pennsylvania Middle District and what it is it is a way in which there are 16 courts and then it's up to the individual judge within those courts to contribute or put their written opinions into this court web so there is a way in which you can if your particular court or the court you're interested in is in in court web where you can do a quick search you can do more thorough full-text search but again if you want to cross jurisdiction and do some searching you certainly can do it through court web you can also just pick out a particular judge whose decisions you want to see you do get a search result this happens to be a particular case the belong be Jerome case and you can bring up I did not create a slide that that showed the actual opinion itself other choice is is of course through pacer and again it's been explained that you do have to register for pacer and that there are some fees involved but there are no fees for the text of written opinions so again the category of written opinions is is can be quite broad and it's not just those documents that we're used to seeing in the federal reporter or the federal supplement or again more the more traditional documents that we see even on Lexus nexus in Westlaw so you certainly have your link to pacer you do have to log into pacer I chose just for this demonstration to look at the US party case index and I know that there are going to be a little bit of information about the new search ability in the US party case index and in this instance again we chose to to look at the jurisdiction of Rhode Island and Judge Smith was involved in the case of that in I'm sorry he was the judge in a case that involved the town of narrow Gansett we did in fact find the case this is just following through as you're working through pacer use you bring up the the docket report you run the report and then you can actually get now a sense of also the entries in the the full docket report you know you do have to search a little bit to find the opinion in the order but it is there in full full text this just shows that you're not being charged if you're going to bring up this document and here in fact is is the document that is available through pacer fee based research services I'm not going to dwell on these we know that there's Westlaw we know there's Lexus nexus Bloomberg law is now also in the business lowest law is available Lexus one there is a free case feature within Lexus one and so again there may be other fee based services but this is the core fee based services that we generally talk about other sources these would be free sources on the web find law I'm sure hopefully many of you have have heard of fine law because it also does cover state cases if you go to the fine law site you choose browse cases and codes again we're looking for the district court decisions from Rhode Island and do you work on your way down and you will find the text of the opinions what it does what all of these free websites do is to take you to the courts website and then you can find the actual text of the decision justia is another source that you can use to get to federal court decisions or cases and again this is just the sort of path that you would follow and in fact this was when from fine law from justia when I got to the link to the Rhode Island district courts website and I ran the search of town of Narragansett near URI University of Rhode Island and so I was able through to this query to come up with this result and in fact this is the case that that the case that was highlighted was the case that we were looking for in for the sake of brevity I didn't make screen captures of all of this other sources the legal information Institute which is free for years has really been working to bring all kinds of primary legal documents out into the onto the web and available with a reasonable search engine in order to to find the cases alt law as another source I I didn't bring it with me but if you go to the alt law homepage they had a statement about they felt that they were done with what they had set out to do and that you know they were really just a very small group from Columbia Law School they had done and accomplished what they wanted to do and so I don't know what the future of alt law will be Google Scholar we all know that recently Google Scholar has added opinions and so that is another way in which not every single opinion but certainly it's another way in which we can get precedent is no longer available I don't know if any of you knew about precedent it really was only in existence for a very short period of time it looked pretty good but I believe that they they are no longer in in business so unfortunately we don't have have that alternate source to look at also this is just the US courts dot gov if you do want to get any to any of the court websites to get to their opinions you can come here and you can choose whatever jurisdiction you want future partnerships in terms of getting the the opinions out onto the web in a more authenticated version maybe through GPO Fedsis the federal depository library program there was the Pacer pilot that we hope to start again on there could be a partnership with the law library of Congress and also there could be more partnership with Google Scholar so there really is an effort I think by the courts to try to get out onto the web available to not attorneys but really in many instances to individuals who are representing themselves in court a lower cost or free way to get to opinions it will it will take some time but I'm sure that we will accomplish that I'm going to turn it over to Judge Smith all right all right well my role here is hopefully just to give you a little bit of a picture of where the future of opinions might be going and to give you a maybe a little window into that and then talk just very very briefly about how that might impact the system that Susan just reviewed for you so typically we think of opinions as written documents and solely composed of the written words the advent of the internet over the last 10 years or so we've begun to see creeping in references to internet sources and that's typically done through what we know as hyperlinks or or hotlinks and thousands of cases have now that are reported in the federal system have various kinds of hotlinks in them I'll get to that in just just a moment the second thing we're going to talk about is sort of the newer phenomenon is audio visual that has been housed on a permanently housed on a court server and specifically I'm going to show you what the US Supreme Court has done in that regard in a case called Scott versus Harris and then finally what I think is sort of the the new generation of opinions which I'm going to show you as a part of an opinion that I worked on I wrote and developed along with the help of it folks including Wendell folks in my court so and that involves embedding audio and visual inside of the opinion to create a multimedia opinion that is not just a composed of written word so with respect to hyperlinks we all have seen this Susan in as a great librarian managed to pull up a number of opinions she keeps a list of this sort of thing only a librarian would keep a list of this kind of thing and there are thousands literally thousands of opinions that I had a walk or do a search and I came up with a thousand hits for WWW so we know that that's going on going on a lot out there but of course this works as long as the website stays stable and and whatever it is that you're linking to is still there a year from now I pulled out an opinion that I had put some hot links into just they weren't central to the outcome of the case I just wanted to check and see if they were still there I used YouTube and my advice is don't link to YouTube because things that are there one day or not there the next so as I said it wasn't central but still even more stable things could disappear and not be at these sites so that's a little bit a little bit unstable may not be the right word but you you're not going to be assured that whatever you've linked to is actually going to be there ten years from now when someone is looking at the opinion now the next generation I think very interesting development was on the Supreme Court in in a case that involved the car stop sort of a debate between the justices in the majority and Justice Stevens who was writing a dissent about what occurred in the in the car stop the court put a footnote in that that said Justice Stevens suggests that our reaction to the videotape is somehow idiosyncratic it seems to believe that we are misrepresenting its contents quoting the dissent in some the factual statements by the Court of Appeals quoted by the court were entirely accurate we are happy to allow the videotape to speak for itself and it cites to the record exhibit and then gives you a web link that takes you to the Supreme Court's website so this is kind of interesting if you hit this web link you put it in your in your search bar it actually will begin to download the video for you and I had a little trouble getting this getting access to this the first time I did it but was eventually able to to view the video the video will download to your computer and what you'll see here is not the entire video but this is what you'll get as you this will be on your computer this is the tail end of the car stop chase all right so what's interesting about this and my view is you can see the obvious power of having a video with audio that you're able to view as you're reading the opinion and and just think about the difference in terms of how this explains what happened versus actually reading the judge writing about what happened the car chase it ran into the back of the other car it went down an embankment it crashed etc etc this is obviously much more much more powerful but the technology that is the what the court did here isn't perfect it it it takes the video it houses it on to on to the Supreme Court's own server and you have to then link to that server you can then download it to your computer but I think it's a it's a it's a step above linking to some outside external source so it's a move in the right direction this was to stop 2007 now I had a case involving a patent dispute involving Microsoft and a technology called product activation if any of you have ever you know installed Microsoft Word or Microsoft Office in a computer you might have gone through the product activation process this case has been presiding over for about six years it finally ended up in a jury trial the jury rendered a verdict in favor of the plaintiff unilock for 388 million dollars and there was a there were a number of issues in the case but one of the most critical issues in the case was whether the Microsoft technology which is known as MD5 or sh1 which are what are called one-way cryptographic algorithms and what a one-way cryptographic algorithm is I'm sure most of you know this but I'll just maybe I should turn to judge Leonard judge Hayes Hayes to tell us we're gonna get to that so in in words what this algorithm is what it does is you can pour any amount of data into one of these algorithms and and what you will get out is a 128 bit the MD5 is a 128 bit which I think is 16 characters string of numbers essentially and and any amount of data can go in the contents of the New York City phone book feature length films the entire contents of a public library can all go into the algorithm and out will come a 16 character digest now it's one way because you can't go back the other way and figure out what the contents were that you put in so the question is in the case whether the one-way cryptographic algorithms of MD5 and sh1 were those summation algorithms or a summer or an equivalent all right now the jury said yes and on a motion for judgment as a matter of law I overturned the jury's verdict this is a schematic a simple schematic of the the product activation technology and I'm not sure you can fully see it on the screen up there but the the issue is whether where you see the MD5 hashing algorithms on either side of the of the of the system on the remote side and on the Microsoft clearinghouse side they have to be the same and are they performing a summation a task this is a another schematic this is both of these are actually in the opinion that I wrote this is the description of the of the patented technology which is the summer that is a series of numbers added to each other and they equal a registration number so the question is is this what was in the patent is that violated by the the Microsoft technology that is the MD the MD5 and the sh1 and I found that the MD5 was not in the sh1 we're not sh1 I should say we're not summers or or their equivalents and the the problem challenge that I faced was how do I explain this because it's very very complicated so during the trial uncontested testimony from a professor computer science professor at Rice University described how these algorithms work and he used an animation that was very effective and that computer animation was shown to the jury through a laptop using utilizing the the courtroom technology that were were very privileged to be able to to use in the federal courts this animation was very effective but he explained the animation as he as he went through it so the problem was how could I use this this animation and I thought that would be more effective than explaining how the algorithm worked in in words only so I in the opinion there's a description of of the algorithm and you come to a point where you see this screenshot this is the first screenshot from the from the video animation the professor a wall accused in his testimony and as you're looking at this on the computer that's one more page you look at this schematic this is a schematic of MD5 so what I did along with as I said the great help of Wendell at the AO and my own IT people in my court we took this this animation and then we married it with the digital audio of professor Wallach's testimony during trial and we created a movie and then we took the movie and we embedded it inside of the the opinion so this is actually in the opinion this isn't a link to anything it's not on a server and it's not in at the court or any external source it's actually inside of the opinion and this is what you get when you're looking at the opinion on the on the on on your computer what's the procedure the next slide so this is the voice of you're now going to know I'm showing you professor Wallach functions these are mathematical functions expressed in terms of bitwise logical operations but let me explain that so what I'm trying to do here is show that waking up that's logical and MD5 is not just a summation it's not just addition shape a logical or and the one that looks like a line with a little hook down this part that's actually a knot and the final mathematical operators and this goes on for us inside of a circle this is about a 12-minute movie or that runs through the entire operation that you're just going to get a six flavor rounds you see at what he's doing show then the next 16 years G the next 16 years H the next 16 years I what's important about all of these functions is all of them are expressed in terms of three inputs X Y and Z so three bits in three choose values in all right so I think you get you get the drift the explanatory power of that video is so much greater than what you could achieve just with the with the written word so what about looking at the the opinion and how you get to it what I found in just testing out some things is you the best way to view it is obviously through the court's website Pacer you can get to it but it's pretty slow because it contains so much memory and it's really totally inadequate with Westlaw and next Lexus Nexus this is what you get when you get to go search this opinion on Westlaw you find the opinion and then where the movie is it basically says that it's an it's an image a picture and you have to click to view the picture and you will get the this is the Lexus screenshot you'll get the picture but you don't get any of the video so just to wrap up I think we're entering potentially a new generation where not just use of weblit hotlinks and server housed video but even video and audio embedded in opinions is really going to change the way we're we're going to create opinions the reason we can do this is all the advancements in courtroom technology that we have been able to enjoy over the last few years digital audio which you can hear more about and the in the video but we need to invest more so that we'll be able to access all of this and utilize it to its fullest potential both the government and the private services so that's it thank you will thank you Susan now we're going to move to talk about digital audio window do I need to switch seats no I think we're okay I think we're okay just a second good morning for who's unaware digital audio is a method of taking and recording information using computers in a sound system I have it pointed this way because two of the judges that were involved in the pilot are sitting to my right so I'm hoping they'll chime in and interrupt me in anything I get incorrect as well since 1999 digital audio has been a method that was approved for taking the record in federal court those courts that use digital audio allow the public to get a copy of any audio of any hearing as long as it's not sealed for cost of $26 this pilot is really an extension of that of that access that's already available the pilot was approved in March of 1997 and in July we were posting audio files now I know that the mantra of this conference is the state courts of the birthplace of innovation but to go from authorization to implementation in four months that's kind of cool we had five courts that originally started in the pilot and we had two that joined us later in the year and we had some initial policy questions when we all met at the start of this the ones that are applicable here are which audio file should be made available is there an interest in this type of file or record from the court and how should action be handled well it was very early very early was evident that the decision of what auto should be made available had to rest with a presiding judge we also were able to determine that because we had disparate digital audio systems in the courts the audio we make available would be an mp3 and the auto we're making available would be a copy of what the court had as their official record so the audio file that we're making available on pace was if for convenience for the public it was not the official court record sorry and redaction the courts followed pretty much an all or nothing focus with redaction if there was sensitive information that was provided in the court hearing the audio file was just not uploaded the best practice was to excuse me remind attorneys beforehand that they should not introduce that information now someone may say wait a second I heard Peter Martin talk about an audio file actually heard the audio file where judge mentioned the name of a minor from the digital audio pilot if you heard that presentation what you didn't hear was that for a year preceding that the plaintiffs and their attorneys had changed the caption the case to remove the initials and actually list the minor by name and they did that in all their pleadings for the year leading up to the hearing that was played in that presentation so in that case it was information might think was sensitive but the parties in that case had made it very clear that they did not consider sensitive information and that's why in that particular audio the judge would have posted it even though he had mentioned the minor by name let me just say this all or nothing aspect of posting the digital audio on pacer for access through the docket is not necessarily all of the day or all of it all of a morning for example in bankruptcy court we have many days that we will hear a hundred or more cases on motions hearings in the span of a morning session say and the technology that Wendell's worked so hard to get in place will divide up the audio files by case number so that each case number each case separately has its own maybe thirty second maybe five minute audio file attached at the at the hearing spot on the docket and the and the judge will determine whether or not he or she will post that one for that case not not affecting all the other cases that came that day which as to which he or she will make the call as well because you can use your discretion to determine what a session is so for instance if you know in a trial that's ongoing you have an hour a sensitive testimony coming up that you just can't avoid you can break your sessions to treat that as a separate session not post it but still post the prior two hours of that morning thank you as we went through this we found there were three areas that we needed to raise awareness the first was with attorneys and that involved reminding the attorneys that the audio files would be made available and the way we reminded them where they were minded in court they're also notices posted excuse me outside the courtroom and excuse me and also at the council tables the second group that we want to raise awareness with was with the public and with the media and that happened rather straightforwardly sent out press releases and that was I don't say promulgated by word of mouth and at least one high profile case in Nebraska the local station downloaded the audio file file from Pacer put it on their website mentioned it on their news station and directed people to go to their website to hear the entire hearing after playing just a clip during the news show and thirdly we needed to raise the awareness of court staff this was new for some of us as we went through this actually knew for all of us in one instance we had a hearing where a judge was looking at reducing the time that someone's being incarcerated they had been a cooperating witness and in this case the defendant appeared excuse me appeared from jail telephonically and after the hearing was done the judge and the courtroom deputy left the courtroom and allowed the attorney to talk to his client so that he could avoid a trip in they could talk about what they wanted no one turned off the video that's giving the audio so that was heard and so we changed some of our internal practices we don't stop the audio until the judge is just about to enter the chambers we make sure we turn the audio off those are things that we had to have as court awareness so that we weren't providing more information than that was actually on the record or should have been on the record we found a number of benefits as we went through this that went beyond just providing more information to the public for the courts there was additional transparency and for court staff it was easier to access a recording because they were used to getting it off the docket they were on the docket all day and that was a very convenient place for them to find it there was benefit for judges I'll I get to speak for the judges here including in addition there was additional transparency there but it judges use it to refresh the recollection I'm told and also to direct counsel to direct counsel at times that they need to refresh their recollection before they came to court or we're going to submit a proposed order there were benefits for the bar attorneys were downloading these audios listening to them on their iPods I'm sure there was some counsel who was downloading it saying yeah I sound good there but also allowed them to be prepared if they were substituting for someone else to know what happened in the hearing the last time they were in court benefit to the media was that it was readily available and we had cases where judges heard their voices on the radio as they went home these were from that day's hearings the media downloaded them and we're playing a snippet thereby adding more transparency to the activities that went on in the courtroom I would say that judges got a problem with the presets on his radio the benefit of the public is that there's now an easier and more inexpensive way to hear what happens in the courtroom and we had attorneys who said their clients were not appearing a quote with them but we're checking up on them by going on and listening to the audio file so they could hear how their attorney represented them in court we also have some law professors who would download the audios and could actually use it as a teaching mechanism to show what actually happened in the courtroom as they were teaching the students have a couple of real-life exam or real-life examples that I'd like to mention during this pilot we had the Chrysler case come up and within five days of having the judge say that he would like to participate in the pilot we had system spokes from South Carolina go up to New York City I'm sorry yes judge North Carolina Eastern District please don't tell them I said that and they went up within two days had the system installed up and running tested worked with the staff and that ended up being very successful that court there are only 30 audio files they started with the first hearing that GM had excuse me first hearing Chrysler had between the two cases or 30 audio files they were downloaded over a total of a thousand times and we also see calls from attorneys saying can we post this on our site so that our clients can go there and hear it and of course they could we also had a case regarding Santa Fumo in Pennsylvania this was a I guess I can call a corruption case since he was convicted it's no longer a ledge corruption case and that case received also wide interest especially in Pennsylvania when I called someone from the court and asked them what would be memorable from that case someone who heard almost all of the hearings they suggested I choose this clip to play for you yes we're at Green Street where they're back and cleaner yes what kind of back and work and how many were there one of the floor and what about at the at the other location they were there so yeah same back and plan yes Florida yes how many were there to Kenyan Avenue and I mean there was two there as well and what about that at the docks I'm not sure I was just there at the workshop maybe okay now it's not just that he had a lot of vacuum cleanser other issues but this was what they remembered the fact that he had the same vacuum cleaner everywhere and was using the credit card to get it but this case also had another twist after the case this case went on for 70 days after this case had been going on and the jury was deliberating it was determined our emotion was filed to stop deliberation because one of the jurors was posting on Facebook that they were deliberating and I have in front of you the is the order that was filed the document for that but what they did is they hadn't in chambers hearing to discuss this and they also put that on Pacer so I'd like to play a clip from that because this to me was just intriguing that I would get to hear what happened in the chambers because obviously the entire galley could not have gone in there do look like he's at least reporting on the fact of deliberation that he's what that he's reporting on the fact of delivering so that they are deliberating what I want to discuss gentlemen ladies is how we go about this procedure today of interviewing does anybody have any suggestions about well before you answer that question let me say this my thought was that I was going to bring them in here and a very general sense of telling why he's here and then invite questioning from the attorneys now if you don't want it or I won't ask you a lot of questions but the thought is that my thought was that because of the technical nature of this which I think I would say that the attorneys might want to ask questions do I know the hesitance in that our preference would be that you're honored to do the preliminary questioning and then we will follow well that's man say first for the record that our position should be in court even now even though we're on the right this is such a obvious public interest and importance and that is my my opinions that's obviously wrong because I think that your deliberation remained the most secret and sacred thing honestly not the so it goes on but then it was made available there is one other clip that I like play for it's about 30 seconds and it kind of gives you a feel that not everything that goes on in the courtroom is completely serious and if you're going to listen to these are the files you'll never know what you're going to hear well this is something that judge Haynes and I have been heavily involved in and as we sit here today we do believe it's on the brink of being approved by the judicial conference coming out of the pilot stage in a couple of weeks so that any court in the country the federal system that uses digital audio can begin to make these recordings available we could talk about it all day I think it's absolutely revolutionary in terms of the look in the federal courts that you're going to be able to remotely have but we'll see where we go and I'm sure talk about it in future years window you want to talk a little bit about the part due party case index want to go fast because we're I think I can do this one fast we have a U.S. party case index which you may have seen you saw it in earlier slide which receives about 200,000 queries a day the system is very good at finding an individual case it allows you to look across courts so if I were wanting to find a case with Wesley Snipes criminal situation it gives me a list and if I were to click on the third case down I would get to see the docket for the case where he had a criminal case in Florida the thing the party case index isn't great at is giving me a list of cases based on just a date range if I were to do that I get a response I'm not looking for so in the words of Will Smith old and busted new hottie this is a new United States party case index and I can still get the same list that I would if I were looking for Wesley Snipes case but there are other things I can do here as well it has more search features I can search on all of these fields and I'm going to if I were to want to get all of the cases bankruptcy cases in the country for the first five months of last year I can run that query and it will come back with 400,000 cases for me I then have the option to download those if I wanted to and if I decide I want to download them it'll tell me the the pacer tries that it would have and it gives me three options that I believe weren't available for I can get them in XML I can get them in text or CSV I'm not quite sure what CSV book looks like but I know I've seen it on some equipment I've had the other thing I'd like to mention is on the party case index if I run that same query for all the bankruptcy quotes in the first and second circuit I get a much smaller number 40,000 cases or excuse me 34,000 cases but I can filter those results and I can look at the courts that are making that up and if I were to click on one of those it will allow me to see just a list from that court and that's coming out in the very near future we'll keep our fingers crossed for March 15 thank you announce the date thank you now Mr. Shackian is going to run us through the user survey we're coming to the end of Pacer over the years has always sought the advice of its users and we have convened work groups and task force all around the country more or less informally this is the first time we've actually gone out to an outside consulting firm and to get an unbiased and objective view paid them a fair amount of money to go out and look and talk to our entire user base and come back and tell us how our users see us and it's not quite done but we have some preliminary results that are worth looking at right and I'll speak quickly Pacer has been around for a long time it's over 20 years old now we've grown we've grown up in 1988 from 12 courts in a bulletin board system it grew slowly at first 9,000 users and as you've already heard we've hit the million registered account mark oh yeah I think it's not logged in here is the problem okay perfect so you can see clearly how much we have grown and that's just the last 11 years Pacer how big are we we provide are you seeing that I have a clue how to get that off of it did you capture you must have captured okay well it provides access to 30 million cases over a half a billion documents and 700 million docket entries as you can see our usage by court type back in the day it was mostly bankruptcy now we're nearly a 50-50 split maybe this will work why do I apologize all right how big are we how many calls do we get we we register about 3,000 accounts a week that's growing to 5,000 5,000 emails a month we take 135,000 support calls annually at our service center so we have this assessment we want to know our users better we don't even categorize them today we wanted to gauge their satisfaction and we wanted to know what more they want from us we had several different research methods we used including qualitative research with 234 key users some of whom are in this room we did a demographic survey we to find out more about who our users were how they're using Pacer we did a service center survey for four months everybody who contacted the Pacer service center was invited to take either a telephonic or an email survey we went out with what we called the mini survey the proactive survey to everyone it was up for everybody and that was all 350,000 of our active accounts and then finally the Pacer user satisfaction survey that was a random sample survey so briefly on the results who uses Pacer well as it happens 75 percent of our users fall within the legal sector or the lit against themselves and then followed up by 10 percent of commercial businesses and then the rest of the user populations which are pretty which are pretty small what do people use Pacer for a whopping 76 percent are following a case on Pacer and nearly half of our users are searching across the courts or searching across cases what were our strengths and we started with the qualitative research I'll tell you this pretty much followed right through all of the survey tools we had but were reliable stable and flexible and we are the complete authentic and accurate source and I want to take a minute you know everybody talks about Pacer I think the real important takeaway of Pacer we're talking about court records we're not talking about copies we're not we are except where it's very explicit like with digital audio we're talking about how the public accesses court records and that's recognized we do have areas for improvement we acknowledge that and they were made very clear to us that these also followed through we get it our user interface was was put out there over 10 years ago and in internet time we know that's too old we have search limitations we need more consistency across the courts and that's going to be a little tricky and frankly we don't do our best at communicating and we're going to we're going to work to change that the proactive survey results we're pretty excited by this because this survey was widely batted around and frankly was advertised from what we thought were some of our less exuberant users as it turns out 86 percent were at least somewhat satisfied with with Pacer itself they also again the results tracked right through it's a fast efficient way to get information it provides the accessibility however there are changes to be made once again you'll see the user interface there was a nod to the pricing and billing the search is right there the standardization consistency across courts this was now these are the random survey results a little bit different you can see that our our our biggest fans are in are the creditors and the service providers to the legal sector the legal sector themselves and the commercial businesses are pretty happy you can see that our highest dissatisfaction this was interesting as among the educational and research institution and students that's not shocking because frankly the system was developed for the legal sector so there you have it they suggested a number of enhancements again they follow expand the search capabilities you see we see this over and over and over again and they were specific we took the comments and these have been summarized but we've been very careful to pull out the comments so that as we make our changes we have specifics again these are the comments I was just speaking of on the ease of use we've pulled out the very specific things we need to start working on this is what we call the ease of use or user interface the search capabilities some of the suggestions for additional standardization or consistency across the courts the Pacer service center for those of you in the state courts who are providing direct support to your users apparently what we found a key driver of satisfaction was if folks are using the Pacer service center not all of our users do by any means they are happier than the people who aren't using the Pacer service center so we say use the Pacer service center if you're not happy they'll help the other key driver we found was frequent users are happier users because of course they're more familiar with the system now since Pacer really are the read accesses to what we keep in the courts case management systems we can start making changes to Pacer now we have started already the party case index that we'll be releasing on the Ides of March of all days it's a step in the right direction but that's going to be they're going to be regular updates at various intervals however I hasten to add there's only so far we can go until we retool the underlying system that'll turn over Judge Leonard well I kept my section for last so I could delete it which will be necessary but I do want to say about two minutes worth of where we are the federal courts is involved already in this internal and massive effort to scope out and build the next generation of case management software realizing that our current version although robust and working well and we don't have a crisis we'll get outdated there are new technologies there are new efficiencies we want to stay ahead of the curve and we essentially have three big different groups working on it we have groups of internal clerks office folks working on the internal business processes we have very vigorous groups of judges who think they got left out last time when the system was designed trying to document exactly what chambers needs and then we have a third group that's called the additional stakeholders which is everybody else who uses the federal courts what do you want from our case management systems and we plan to spend a good bit of the next 18 months eliciting from you in every way we can think of even tweeter to get as much information as we can about how you use our systems now what you see its deficiencies as being and how we can improve and I happen to chair that third group so it's going to be a large part of my life for the next couple of years now we are right at time but I would like to give you an opportunity for a couple of questions there's just so much material that we could cover we've tried to be concise and I hope you found it helpful and if you've got anybody's got a question I'll be glad to try to take it Mike Johnson from Minnesota we use digital audio recording through a vendor called court smart but the systems are so sensitive they will pick up every conversation in the courtroom and there's concern there that conversations between council and client even though mumbled and muted so that you might not hear them from the bench or even from the back side of the courtroom are being picked up by the system how sensitive is the system that you are using and have you encountered any of the same problems well we use all of this all the systems I think every proprietary system is used by some federal court and part of the complication of this pilot was creating an interface that looked the same that worked with all of these different systems so some are more sensitive than others that is a real concern you have to train your lawyers to step away from council table or turn off the mic I mean the mic at council table has a switch we tell them if you want to have a private colleague with your client turn the switch off so that it doesn't get picked off the picked up the ultimate protection is if we do pick up something that's totally inappropriate and the lawyer lets us know that that might have happened we just don't load up that session in my courtroom we try to be as insensitive as possible but you're the goosing court anyone else up there I'm sorry where did you point I just saw a hand right here hi Lisa Norris Lampy from Oregon I had a question about the written opinions am I correct in understanding that certain orders of the court are not uploaded to pacer because a judge has determined that it's doesn't qualify as a written opinion or is it just that it's tagged as a written opinion and that enhances the search capability I'll take that pacer is a view on the entire case case file so any order entered in a case unless it's sealed will be available on pacer there's a way they can flag particular orders to be opinions and we have a special written opinions report then that you can run and they show up on that special report and that's a function of the e-government act which requires court opinions to be available in searchable mode at no charge or by statute so that designation triggers search from the website and free access to the public Steve Schultz can you speak a little bit to how the courts are funding this next generation of CMECF and can you also speak a little bit to what the fees what the pacer fees go to fund it's your project we will we charge the electronic public access fee to and it all goes by statute right back into the program one of the things that we we expanded the fee from seven to eight cents a few years back in anticipation that we are well a couple of things we needed to make our current infrastructure more robust into we're replacing all of the servers in the courts right now we're we're completely replacing our network and so essentially it's all it's a working capital fund so there was a lot of discussion that we had surplus not so we had caged some money off so that we could make these infrastructure improvements replace our network so that people actually could use this functionally and as you can see with the digital audio and especially with the written opinion we do need that new network we're working on that the other thing we've been caging money off for is to fund the next generation of CMECF which is considered part of the electronic public access program proper i'm going to make one more comment on fee revenue there is certainly a public perception out there that we are ungodly expensive we don't believe so in fact another what we hope is a big announcement what we're waiting for right now if a user doesn't accrue ten dollars in usage charges in a year all fees are waived they're not billed that actually tabulates up to 50 percent of our users have not did not pay a bill in 2009 did not pay a bill in 2008 so fully half of our users don't get a bill today we went to the judicial conference cacum did and we're seeking authorization to make that annual waiver of ten dollars a quarterly waiver of ten dollars uh should the judicial conference approve that they're meeting this month we estimate that fully 75 percent of our user base will no longer pay a bill and just with regard to fees the one thing i think we should all keep in mind and you can dice the numbers up a number of ways but it's not that people are paying for the information the information is free at the courthouse as it's always been in a good portion of it based on this written opinion directive and based on the waivers that michels has talked about is free at your home at your desktop but what you're paying for is the delivery system uh and maintaining the delivery system it's not a price for the law it's a price to have it handed to you on your desktop at your convenience at your command and to conflate that pricing with a price for the information as opposed to a price for the delivery is get you off on the wrong foot and thinking about these charges but let me in with you absolutely go ahead from the standpoint of the trial court um these fees also go to funding uh courtroom technology improvements and i think the uh i think the amount of investment in courtroom technology uh in o nine was around 25 million dollars that courtroom technology has a number of important sort of public access components to it for one it makes available all this digital audio we talked about and that thing in in the case of the opinion i showed you goes back into the written uh has the ability to go back in the written opinion it also provides the technology that allows sophisticated animations and expert witnesses to do their thing in in very complicated trials and we've got a problem out there which is that juries we're we're selecting juries who don't have frequently the capability to understand a lot of very very complicated technology with this computer technology or chemistry or molecular biology and we've got to find ways to help juries understand this stuff and courtroom technology does that because it allows witnesses to prepare animations which are shown on flat screen monitors in the jury box every juror has their own flat screen monitors and it also allows we just went through a big upgrade in my court courthouse my courtroom one of the things that we've done is uh large flat screen monitors which will now and this is a very historic courtroom so it has to be done in in accommodating the the historic nature of the of the courthouse in the courtroom we have flat screen monitors now which will enable the people sitting in the gallery to see these these animations and that are displayed so they're not reaching out leaning over trying to watch it on the council table monitor as well as audio enhancements in these big courtrooms with 30 40 foot ceilings where audio gets lost we spent a lot of money on audio so the people can hear what's going on we've just put in new audio that i've never heard of this before but it actually embeds the speakers inside of the benches in the back of the courtroom and inside council table so that the the the wood benches actually perform as amplifiers so now the the back of the courtroom can really hear what's going on this all ties ties together and it's and it's funded through these fees so i just i want to make the point that this is an integrated system it can't be looked at it's just just one thing we are approaching 10 minutes over and we're going to intrude on the other panels so we can have to continue these in the hallway i think they're interesting topics we're not perfect i hope we've demonstrated to you that we are enthused and we are involved and we are trying to be thoughtful as we move forward thank you for your time