 So, as you can see, our population is dropping off a little bit, which is fine. After 15 of these workshops, it gets a little long. So, we have two more hours, and we're going to like take it as long as it needs to go. And if we get done, we get done. So, there's no reason we have to keep this going to a three. And we really have free things to get through at this point. The technical principles, the un-technical principles, and the what's next for law.gov. And so, we're going to let Tom Bruce, with a degree in theater, and the technical principles, I'll do the legal principles with my degree in economics. Sounds good. So, nobody up here knows what they're doing, and the vast majority of people that have wisely chosen to take their bag lunch and leave. So, let's talk about... Well, you will recall that Carl mentioned this morning that at the core of the document that he is going to write and circulate as the report on this effort, there will be ten principles, five of them technical, five of them, as I would say, more policy-oriented. I'm leading off with the boring ones, which are the five technical principles. But I think we already know from this morning's discussion that even technical principles as bright as they may seem very quickly shade over into policy in this world. And sometimes they can be useful ways to disguise the fact that we're making policy prescriptions. But in any case, they're a little juicier than they usually appear. And Carl has proposed five that I will just throw out here in the list in a very summary form, and we can then take them one by one. Those are availability via bulk access, authentication of access, and access methods and documents, availability of historical archives, development and use of interoperability standards, and vendor and media neutral citation. None of these I suspect are particularly new to most of you, but we'll look at them in detail shortly. What I would like to do and to discuss over the next 45 minutes are a number of questions about them, right? So first of all, have we covered with those five principles everything that we think is important? Is anything missing? Are there things that should go with the supporting material that is developed to surround this, some of which I suspect will emerge from this discussion? Some things that I thought of that just on the face of it I would want to see along with bald statements about these are some thinking about whether there's differential treatment of any of these, when we're talking about case law statutes and regs, some of them seem to me more or less applicable to one or the other of those. What a standards development process or perhaps more accurately lack of process might actually look like when we're talking about standards. That sort of hits the low hanging proof first. Can we at some point want to bring in an explicit discussion of costs as being either sort of implied or alleviated by any of these? I have to say I will tip my hand a bit here, and I don't want to start World War III over this just yet, that the principle thing that's driving me on a cost issue is actually the question surrounding authentication, because I think that frankly most of the schemes that I've seen put forward so far don't necessarily play well against for example historical archives when we start talking about what the costs actually are. So that said, let us look at, actually you know what I should do, I should go back, I should go back to just the list of five and start by asking is there anything else by way of a technical principle that anyone thinks should be on this list? So again we have availability of stuff by bulk access, authentication, the idea that we need historical archives to the extent possible, interoperability and citation which is in some ways the specialized form of interoperability that everybody would laugh at us if it wasn't dedicated. Do privacy concerns go in either of those, the technical or the non-technical categories? Yeah, the non-tech there is an active program of R&D with a specific call out for we need much better privacy tools. Because somehow that seems like a very technical problem to me, rather than, you know. Well the problem is there's a solution, there's no decent tools out there to do privacy as one of the big roadblocks. What about the program that you just read on my computer last week that pulled out that made a list of every social security number or miswritten phone number that had not been digitized? Because that was like Adobe Acrobat Pro? No, it was some other bizarre thing that the university was doing to all of us for their... At the level of 10 and 20 million page archives, I have yet to find a decent open source solution that effectively does not only social security numbers, but does more heuristic things like names of minor children. But it will definitely be mentioned, it's definitely one of the issues. Privacy has to be in there. Canadians have done a little work on this area. It goes well on sort of prospective stuff. The expenses always... The cost model changes considerably when you look at the perspective of a bigger library. So I have one... And again, you have a semi-blossed version, since I've seen it slide deck already, that goes in a little more depth. But when you say availability via bulk access, the crowded places like Sunlight Foundation would say we also need notification mechanisms. We need to know what's new. It has to be easy either through a mechanism like Adam RSS or even just simply a directory listing to know what's been added to the bulk archive. Right, so some form of current awareness, automated current awareness services is certainly a piece of this. I suspect that actually there will be many of those, not only around specific technical standards, but also around different user communities. For example, I mean, I've been cross-serving on this subject many times. I've gapped for years about hooking the AI-PMH up to judicial disciplines. You know, having some form of metadata harvest available for judicial decisions, it just makes perfect sense to me as a way of running a federated collection. So there's all sorts of notification functions. We're in the idea of federating these things. Then I think fall under that, that also do service as best current awareness services, yes. Under, again, a subcategory. We were talking about the dates of things yesterday and how people don't date things. And that to me is, I don't know if it's inter-operability citation. Interoperability standards has actually got a comma, if you look at his gloss, and that says such as metadata standards, document IDs, and I believe the date problem is just a metadata problem. They're thoroughly treated under metadata, I think. Christy's early. Key metadata issues is jurisdiction and date. Anything else that they might be missing? Anything anybody sees that shouldn't be there? Oh, yeah. Well, I just, I do understand the argument that says development is a comparability standard, but it seems to me that to me, you have to have at least some, even if there is an agreement on the metadata yet, you have to start with metadata. Availability should imply that then, I say DTDs as if it's an XML, but the description of what's in there, there's lots of things you can get bulk access to, but then you have to decode what the code fields are. This happened to the patent industry. You get a field of the patent, and you have to spend 35 bucks to the patent office to get a faxed version of what that code means, and it can take you a month. I mean, we try this, and they keep, I mean, eventually it's been open source now, but it's that kind of thing where, there's no decoding ring, and I don't want to necessarily push off as a sub point that we're going to, for me, on standards later. Well, remember there's three levels to this. There is the high level declaration of principles, and the question is rather than say interoperability standard, should we maybe say interoperability standards, comma, including metadata and document IDs. The next level, that was the FAQ, like what the hell do you mean by, you know, metadata? I just want to define, I want to say the availability of documented, you know, I wouldn't put it at, and I think if this is a core principle of making data available, is that when you make data available, part of the availability is availability of documents and their descriptions, you know, the keys and the tables. Okay. Standards is another thing that's important, but I don't think, I mean, I think of as the metadata, you know, what we would say metadata, that they might not understand is metadata. It's, you know, the problem. I think point second is the question of flipping an emphasis on, and we hear you. It is absolutely the case that most metadata projects that I know of in any discipline suffer from oral under documentation. I mean, there's a sort of belief in the technical community that if you throw the scheme up there, you throw the DTG up there, well, then everybody understands what the hell you're talking about. And of course, that's not the case. I mean, and I think we can sort of assume, I hope we can assume that what we're talking about here are high-quality efforts and not bad ones. I understand exactly what I'm saying. But rather a bad one than nothing. Now, I think what Zala and I are dealing with is the issue of we've dealt with so many bad, uncoordinated metadata that, you know, part of the message we're trying to get across is you need to do good metadata. But I think it's a worse-than-thing issue. And we hear your point very much and that you want to hear, you know, that documents are important but the documents without the metadata, and I'm assuming the librarians will agree with that point. But, you know, the point that you're making here indirectly and it's an important one is that part of the gloss for this really should be use cases. Yes. And those use cases shouldn't be as far as I'm sort of mean, but the use cases should, as far as possible, stand in contrast to the behavior of the existing system. Right? So, for example, a classic example of the sort of thing you're talking about is what happens when the more private citizen confronts something like regulations.gov that basically requires them to know which agency is regulating the thing that they are interested in because they can find any information about what's being regulated. It's classic metadata failure. Right? So, as I say, use cases that sort of stand in contrast to what's going on with the efficient existing system that they're very important pieces of data. I think in some ways that's pretty working. Yes. What does the poor son of a bitch do when he's looking? So, one of the questions I think you should ask yourself is if we come up with these 10 buckets, could you write a document that explains this particularly important fact? Because what we're hoping is that we're going to get the statement of principles with the FAQ suitable for being some law school members, but there's going to be a whole bunch of other glosses out there digging real deep into some of these things and saying, let me explain in great detail about the metadata problem having to do with the following domains of application and why it's done right, why it's done wrong. And what we're looking for is a high-level vocabulary that everybody can agree on that says the 10 categories of things that are important. Joe Calantrino at Princeton has a series of posts on being a law school member on this sort of thing, or good practices in government exposing the data and ways to do it. So, there's a lot of work there that he ends up with. Yeah, but the big question for you is when you look at these principles, is it something that you can look at and say I wish to explain point number three in greater detail as opposed to I believe these 10 points are all screwed up and here's my 10 points, right? What we're trying to see is can we get everybody to say that the 10 key principles are these? Well, let me explain the one I really care about and it's subtle tease of why it's actually more complicated than just one sentence. And I respect it in many others. I had to restrain, Carl sent me some language on these yesterday, and I kind of had to restrain myself because I realized part of my third reading them that they really need to be a little bit of a workshop test for the person who's, for the ultimate viewer, right? I mean, we want things that can, the consensus can actually develop around it. So they don't want to invite any more logic chopping than they have to, right? So in that sense, they'll be a little bit, they'll be a little bit aspirational. They want to be focused. They don't want to be, you know, mom, apple pie and that sort of stuff. But on the other hand, they do want to leave a little bit of room for people to see and perhaps a little bit of what they want to. Authentication is a great example of that one. We should have authentication. Go on much more than that in principles. And a lot of people don't understand authentication. I had a really enlightening discussion with my dean about that work. He had no clue. And then all of a sudden, he realized, oh my God, all the university recs are posted on the web in a totally insecure way that could be, you know, somebody like me could do bad things to them. But we all want in the principles to have the discussion of the information on whether you should be able to point to any three words and know that they're all fact. Because there's no consensus around that principle. I can, I'll bet you can get consensus around whatever causes them. So taking these, I mean, we've been jumping around and that's inevitable and we'll continue. But let's take a look at that number one, your availability for your bulk access. As written, primary legal materials should be made available for bulk access. Current awareness mechanisms should be built so that data consumers may know when new materials are added. I note that we do not say XML in there, although we are probably implying it. I mean, what else would you use? PDF? Well, I mean, PDF is a fact of life. You know, the bulk access principle basically says I don't care if you've got word perfect and word star files, you just need to make them available to you. And that's principle number one, is what you got, you've got to be able to get it without having to write a web crawler that changes every two days because you changed your underlying code or having to go through a search engine that doesn't have permanent URLs to figure out what's there. I mean, that's that kind of core key principle. And that differentiates it from the, gee, if you put up a website and I've already got keyword searching, why do you even need my data because I've made it available to the public? So we need somebody to generate ultimately a sort of what bulk access is and is not. Yeah, and I think that, again, is a glossy thing. Maybe in the FAQ, there are currently mechanisms available, such as FTP in our sake at a technical level. And then the core point being that you shouldn't have to be a rocket scientist to write a program into some proprietary website. Now, how you explain that to a dean, I don't know, but it's all about repurposing for other people building websites, not just us. Yes? That's not the interview that we talked about yesterday. These various versions of things. I mean, to be honest, what is primary legal material when it comes to the law? What was common in 1995 is... Well, that... ...2005, if you'd be surprised that... I don't know what I want, to be honest, but it's just... So that principle three was availability of historical archives. Does that cover that? Yeah, maybe it needs to go under there. What are we archiving at different stages? Right. So that's a gloss on availability of historical archives. That's where it states them. I think you need to flush that out, because I think most people think it's current on the internet, and that's fine. Well, I mean, the FAQ, for example, might talk about how the Code of Federal Regulations is interesting. On the other hand, when you sue somebody, it's over a violation of the law at some point in time, and you need previous versions of CFR. I mean, let me ask my library friends in the room who have done time on collection development activities and acquisitions groups and that sort of thing, a pragmatic question. So I think you were saying this morning, Carl, about the... And we'll say again, I suspect in a few minutes, about, you know, what is primary in the material we need to say that, and things about this archiving issue, lead me to this sort of much larger question of... Never mind that we all know that it's very hard to set a discreet limit on where this thing stops. How do we avoid entangling ourselves in endless discussions about where it should stop at the point where we're trying to enunciate these principles? This is why I'm asking those of you who have been on collection development activities, because I assume you must know how to do this, right? I mean, how do you do that? How do we set up a process where we don't get involved in endless rambling over what is and isn't, what should be and what should be, et cetera, et cetera? I think that's going to be a bit of a challenge. I think it's up to the individual jurisdictions to be determining that, right? Don't you do it the next way? You just start with the big stuff and go until you either run out of steam or energy or whatever. Whichever comes first. I mean, do you like the way I defined it, which says at the top, we know what this means, and as we get lower, you're going to have to figure out your definitions. I don't have a problem with what you said and I would add one sentence. Please don't lawyer us on this. I think you need that level of ambiguity in the sense, you know, and that having it the way he has it, I don't think you're opening, I don't think the venues you're bringing us into that you're necessarily going to open yourself up to that same kind of... Well, we got out of Colorado, the chair of the judiciary committee was, had to be nuts, putting all the historical archives for every county out, that's just not going to be possible. And I was like, well, you know, agree. It'd be hard. It's an aspirational goal to strive towards. If you can take it into detail, then you strive people away, so I think the way you have it worded kind of is specific and that leaves it open enough to bring to whatever point is appropriate for whatever jurisdiction. I will get immediate feedback on this for the Ninth Circuit, because in my cover letter to them and my submission materials, I define what I've met by primary legal materials, and I said, this certainly means Supreme Court, this certainly... I used the word certainly three times, and I know some judge is going to be like cutting me off in two seconds for using the word certainly, so we'll get some feedback on that slippery slope argument. I think it's analogous almost to the copyright issue in terms of like, you know, the quote there about how it's basically no codes and statutes, and then there are all these other more gray areas. You know, so as long as you've... which I think you kind of did in a sense to find what it's definitely covering, the rest of it is going to work itself out. Well, not go as a shining city on the hill. Yeah. And you know, my answer on the county one would be the librarian. What's most requested should be converted. That's fine. Not necessarily everything today, but... Okay. So maybe under the availability of historical archives, we certainly note previous versions, and in the FAQ maybe we note the principle of most needed materials, or citizen requests, or... So anything further on bulk access before I hit the space bar, make this slide go away for a minute? Okay. Can you do that with authentication, too? Okay. I wish I could make that go away. Well, I mean, you could make it go away, but then everything you do will be of very limited usage, Michelle. Yesterday was complaining about the disclaimer on the mass general loss, but I think that's an essential disclaimer because they're not authentic and vindicatable or dated even for that. Hey, we buy into the authentic. Believe me, we buy into the authentic. No, I don't... Don't misunderstand me. I absolutely understand the need for authentication, and I think it's a fine idea. But I think this is the one that needs a lot of explanation. I mean, my dean was like... He couldn't believe at all that it was an issue, and then when I pointed out what the issue was, he was like, oh, my God, that issue was in a lot more places than we thought. We should pay more attention to web security, to sources of stuff on the web. So for documents to be authenticated, can we do that by example? So in the FAQ media, it says, for example, the code of federal regulations produces XML source code, and there's also a PDF reference copy which has a digital signature on it so that anyone looking at some language that purports to be from the federal register can go back to this document, verify the signature, and verify the language. Is this efficient? I think so. I mean, it makes implicitly a point that I would love to see made explicitly, actually, which is that there are different forms of authentication or different authentication regimes that are appropriate to different types of end-user activity. So often what has come out of the library community on this issue has been premised on a background assumption that people are involved in high-stakes litigation or some other sort of highly aggravated setting. Whereas, in fact, a lot of people are simply doing a reality check with this stuff of some kind or other. They're all lying, doing something that I guess I would call legal risk management. And to drive up costs for that audience makes very little sense to me. The analogy I often use is that there's, sitting in Paris, there's a piece of platinum that we know that it is exactly one year long. And if I am hanging a picture in my living room and want to center it on the wall, it's very solid that I go to Paris to check my tape measure against that bar of platinum. This is not a high-stakes measurement activity. In that direction, it may be as though saying authenticated, it should say authenticated on the ball. Ah. I like that. I do too. I do too. I would change it on the slide if I didn't know. What program are you in? I'm using Keynote, actually. I'm not that adept. No, this is purely operated. Not the program. It has other difficulties. And for access mechanisms, I thought the FAQ would explain why when you go to a banking site, you're using Securities, DTP, and if you're going to the Supreme Court, you probably ought to get that same level of trust. There's sketch more to say about this, or we're going to say any of it now, but I think that's the space part of it. I think at the principle level, authenticatable, I think is a great question. If you want more input, the double A level, in my report, I will be spending at least ten pages going through the existing studies on authentication, best practices on the internet, and citing all sorts of sources. There will be a lot of footnotes in that section. Maybe even some math. So, the story from our premise was based on precedent to the extent possible, which I take to be a non-prosthetic. Complete the story or I should want to say any of the things specifically there about it. Things, one that says, for example, the CFR, previous versions of codes are important, same with statutes and the things. And needless to say, this costs money, one should probably put the high-demand things first if one has to prioritize. And to the extent, if you could mention dates in here someplace, I don't know how. See, it's something about effective dates, not just versions, but effective dates, or to the extent, captureable, whatever. So that's a metadata thing, isn't it? Yeah, although, when we started talking about the archive, that really brings very clearly from the issue that was raised earlier about sort of having appropriate cataloging data on the site. I can see that as an FAQ, that in putting up the archives, it should be clearly stated what the effective dates are of the different versions. What you don't want somebody doing is going into an old version and thinking that's the current law. That's bad, and you don't want somebody researching the law in 498 and pulling up the current version, and that needs to be clearly demarked. Footnote, see metadata. Catalog and labeling these ones. When you start tiptoeing, we're quietly into things like records retention policy, right? They think it would be helpful as we write their bosses on this call, too, to look at all of these from the standpoint of somebody who has a little guy sitting on their shoulder saying, what's this going to cost me? What's this going to cost me? Which in this case translates to how long do I have to keep this stuff? How long do I have to keep this stuff? How long do I have to keep it? And the further you move into the curriculum, the bigger question that becomes, are you really going to keep every driver's license application in the state of New Hampshire going back to the invention of the model tape? Probably not. Speaking technically, do places like Massachusetts where we have, where it's all databases and dynamically generated webpages, does that make additional problems and do we need to address whether that's their problem? Well, that's a bulk access issue, isn't it? Okay. It's not available for bulk access if it's buried in a database and I can't grab it. I don't care if it's in a database, as long as I can get it out. It's not in a database. Yeah. I don't care where you keep it as long as I can somehow crawl it and pull it. Do we want to make explicit mention that the historical archive is as governed by the bulk access or it's pulled as prospective? Sure. I think you're on point three, keep bulk access. Anything more on historians? Yes. That's what you're saying on the driver's license. I mean, we have in the court system records management. We do know what we're going to keep or not keep. I mean, some things we can pull every case or whatever. So there's a part of me is not here, but lower down, do we want to honor local record policies? I mean, if people learn how to pull, I mean, they've already made their policies. Yeah. So a way to win people over is to say, we understand. Yes. Here's the problem. We are going to keep it in bulk. We already know that we're pulling every case and family to keep for archival purposes. We're not going to keep it in bulk. Laudakov is not about making the principles here. It's about saying you should have principles. Right. And a general principle is if it is primary legal materials, archives should be complete. But if it is your local policy, that your archive is complete after throwing away everything more than 30 days old. Well, that is, you know, in too many ways. And if it is your primary, these are more the subcategories than the one under primary. I think that's one of the things I was thinking too. And this is more even more applicable to the slide on interoperability that's coming up is that even if the entire alphabet judiciary lines up on the steps of Supreme Court tells Carl Goh to climb a row, I would hope that the enunciation of these principles would spawn a certain amount of discussion in the professional subcommunities like the core library and so on, about what best practices are in light of these principles. Right. Because I do think that on the one hand, yes, it probably means wholesale adoption of stuff that's already out there that's sensible. And it may also be examination of some stuff that's out there that's not in terms of what people are doing as matters of progress. Because once you start looking at this stuff, it's always seemed to me that once you start looking through this stuff, at this stuff that blends in the public access, then what you decide you want to do about how you handle it changes rather drastically from what it would be if it was just sitting in a law library and just bizarre taking on it. Standards and interoperability. Technical standards for document structure, document identifiers, and metadata, and other aspects of document structure and format should be developed and applied as extensively as possible, consistent with standards under development in the U.S. So maybe instead of a short version, maybe that's a longer version that is what actually goes into principles? Maybe. There's a couple of things. I have some points of nervousness about this and I'm the one who expanded the language. Okay. My first point of nervousness is that I think we have to be very, very careful in this, particularly with respect to document structure. Not to be doing anything that could remotely a thousand billion, billion years be interpreted by a judge that's saying, we're going to tell you about it right now. Because the amount of sensitivity on that point is just unbelievable. Same thing is true of metadata when you get to it. Well, let me make an ethnographic observation. The whole time that we've been working with federal agencies, I'm regulatory stuff which I have been in various guises for the last three or four years. I've been astonished by the extent to what government lawyers confuse the labelling of the standardization of labels for a process with the standardization of a process. And what I mean is this, if you told these guys that for reasons of, I don't know, a great bulk of deal, everyone in the government tomorrow would be required to use green telephones. Three quarters of them would assume that you were telling them what they could say on the telephones. That actually happened in Congress when they gave permission to tweet. But you had to get clearance from the Committee on House Administration before using a new technology. And they said, you can use social media, but you have to let us know which ones they're going to use. And a whole bunch of people seized on that as Pelosi wants to check my blog before I'm able to publish everything. Now, my worry about that one is that they accept our offer and they say, that's great, we do need standards. We accept your entire offer and please come back in 10 years and we'll have the standards done. We're not going to do anything until then. So that's my worry. Get on that point. People look at it and say, every court in the United States together to all agree on what the standards are and when we do that, we'll begin making the data available. Could you just say move towards technical standards in other words, put up what you got or what you can put up and we'll work around it? Okay, so these sound like FAQ things. So, okay. Although actually one might lawyer this a little bit or work with it a little bit by saying for example, other aspects of document structure have been developed and progressively or presumably applied as extensively as possible. I mean, something in there that indicates that you're not going to stop the union of the party and the music's not going to stop all this is going on. It's like authentication is not an answer. It's an ongoing commitment to a process and the same thing with these document structure things is you should get better today and then you should keep on getting better and this isn't going to be a long time thing. There clearly needs to be an FAQ thing about the local authority to decide what it is you need to do. You know, appropriate caveat about let's not have too much exceptionalism. Not too much exceptionalism. Not too much exceptionalism. I mean, there really is a There's still a lot to each jurisdiction and really, you know within the framework of its legal framework to decide what it is it's going to apply. I mean, there's a Goldilocks problem here. There's no argument that's just right. Two things. One, I don't know if it would be worth to do interoperability in standards. I mean, is our first goal really that things can interoperate? Yes. And that may take some emphasis off of the first thing you want to get done is the standards that we want just look at them. And my second thing my sense is with a little municipal government especially probably with a little municipal government I think people would love to have guidelines even today. Why are we having every municipal person sit around and figure out what to do? So standards may be too strong a word. This may be fine for standards but in describing it at some point I would say at least minimally can you just do this, this and this because we don't want there's three hundred sixty-one towns in Massachusetts why are they all sitting around and trying to solve this problem? I agree with you don't say you solve it this way but even to say here are three options why do you think what's best for you and get us moving. Tom has a level one level two way this is a standard computer science thing whenever you come in for a feature for developing software that's a wonderful idea we're going to put it in version two. No, I mean there is obviously a sort of progressive way in which one can address the standardization process not just because you know pretty much anybody can look at a judicial opinion and say okay I can identify ten things I need to know about this thing and you can do all that without having to get locked down and actually have some more interesting questions like well how am I going to use metadata to represent the procedural posture in this case because you really would like to do that it's just that you're not going to do it by next Tuesday and there's a certain amount of stuff that you want to do but you know another sort of surrounding materials thing on this I don't think you can really do interoperability standards and I'm thinking about your 300 some on town center without nodding to some rather specific notion of trying to do public education on this or professional education I think I mean one of the things that have struck me for years is that I actually think reporters and decision decisions are very conscientious very good people and doing this stuff in print and I think frankly that if they knew how badly they sucked when it comes to the electronic world there would be some kind of collective professional suicide right now because they really don't know I mean in a lot of cases they've advocated off the IT people so forth and so on and they just don't know it shouldn't be as hard to write scrapers as it is and it wouldn't be if reporters and decision we're doing in digits what they've been doing in print for many many years very very successfully and so to an extent this is always an education this is now principle number five of the non-tech education and training is a key part of making this happen yeah exactly education training reference implementations yeah this horse dead yet? not yet okay although I'm not sure what my question is but because it's written in the third person it's in the whatever the circumstances it's a lot that got taking some responsibility for putting together these standards coordinating the standards or are you just saying someone needs to do this in the report I will summarize the Cornell workshop which identified a series of existing standards and methodologies and use cases but I think what we're saying is government needs to take responsibility and it needs to work with places like LiI and AAOL and others and there are some existing ones such as as vendor neutral citation that could be applied very quickly but no we are not being specific in any of that stuff even on issues like standardization should happen I think we need to say there are options there are places like the administrative conference of the U.S. but again this is government should take control of their destiny I think is the right approach here this is a genuinely difficult problem in fact when we were working on the ADA committee report on equal means about two thirds of the discussion committee discussion I actually had to do it who's going to make this happen with INGO because there were a number of candidates they were each of them plausible in a slightly different point and it really is a big question if you have an existing standard out there like the ADA citation standard like the URN lack of statements coming out out of Italy for a doctor who identifies like so that a day of work has been done who locks that in to go how does that happen it probably doesn't lock in with that modification but who does that and that's not clear that's not clear nor can we hear anything yeah any more on this work yes I'm sorry I can relate but what's happening with like interactions between other LiI institutes and governments of their jurisdictions is there any lessons to be drawn from those good question costly or valid canly yes and no probably the cleanest example is canly because their relationship with government I think has probably been productive the majority in my view I mean there's been some excellent work done it's mostly been done because it's been taking place in areas where jurisdiction is much less complex there are not nearly as many actors to deal with they all know each other not fifty states not fifty one not fifty one major jurisdictions not a large population not separation of powers not decentralized approaches to a lot of stuff Australia is a very small place so that's an important aspect to bring out but I'm not sure that's relevant to this criteria here but it's definitely an important thing to look at what's going on it is relevant in the sense of there are a series of international efforts and standardization efforts that we should be drawing upon and interoperability on an international level certainly one of several goals let it go that I think we should be striving towards particularly at the level of like national legislation and regulation if it's something like this and you know to use it or document ITEs for example it would be real nice to have the U.S. namespace interoperable with other namespaces that are there and that should probably actually go in your call out of a research project absolutely there's a sitting on those yeah and there's a huge pile of notes from the Cornell two day workshop on tech standards and a whole bunch of references I'm hoping I can find in my pile of materials as when it saves I'll let go before I'm just down I know more on this I'm pressing the button neutral citation this should be eminently non-conversive right paragraph numbers I mean it's just so simple well yeah this actually kind of goes back to a gloss I'd like to see on the standards yeah a little bit I've always had a little bit of an issue with the fact that double A double L stopped short of defining what paragraph is right because what they did was take something that ought to have been a simple computational exercise of detecting two sequential hard returns and turn it into a fifth generation AI project yeah Tom and I have talked about this we have tried to do vendor neutral citation and for example I've got the circuit court opinions I can't do it from the outside if the court were to number the paragraphs then it's easy I just take their numbers but if I number them and Tom numbers them we're going to come up with different numbers yeah it's actually it's actually it's always very tempting particularly for some reason when there are lawyers in the room or for semantics to creep into this in ways that they really should not and I understand that I understand wanting the sort of head compatibility of and wanting a sort of accurate representation of what the document is and that would lead you to consider questions like well does this hanging quote really come as a paragraph you know you would want to look at stuff like that but the fact of the matter is that again for reasons of cost and output ability you really don't want to do that you know you really want to make this mechanical that you possibly can't because it just so maybe the FAQ on that one is is a very polite statement saying the double A double L and ABA standards are of course the starting point and however should reflect current implementation experience of those standards from jurisdictions such as all of them I was pleased to see your use of the phrase consistent with yeah and are you asking the working groups to go back to double A double L I mean maybe at this point you want to go back or is that opening up a can of work you know the issue now is just getting the jurisdictions to want to do it and I mean that's step one but your idea of one number in source doesn't make some sense I mean the concept is right but the implementation I'm pretty sure that the AALL initiative is just to get people to adopt the standards that were written a long time ago and it's just they're trying to get they started starting the last year or the year before they went back the old access to electronic information committee and the citation standards committee the portion was to get people to adopt I'd love to see a two page gloss come out that says let me tell you why double A double L isn't able to be implemented and gee we think somebody ought to reopen that and you know define what a paragraph is but I don't think at the whole level I'm not quite sure this is the time we would work on it I actually my feeling about this is we're at every principle 100 blog posts if people want to if people want to write glosses on this and by all means they should it would certainly be far more interesting to read about the reference test the best thing I'd want is like the reporter of a home at a blog and say we'd love double A double L but we had problems figuring out what a paragraph is and we think you know body X like the National Center for the courts ought to like define the paragraph double L doesn't actually matter I mean as long as you're doing it prospectively whatever they do is what it is if the source is doing it it's easy because then you just do whatever they do right but the problem is you they're not the problem is is how to do the archives right and so you publish your archive and you can bother going through a number of paragraphs and both of us are mirroring your archive when we want to number those paragraphs one of my authors to the Ninth Circuit is actually the ability of handing them a copy of their archive back that has paragraph numbers in it but well except for it makes like two different worlds but what's the problem with just using the old star paging that they used to use in print to print translations in taking prints electronic of all things well there's actually the fundamental problem that a citation is is a page-based citation is not a document and you think about the problem for example with multiple pages per page table cases that kind of stuff it's not wholly satisfactory as a unique document but again you know you can reasonable people can disagree about how big a problem that is just as reasonable people can disagree about how likely it is that there will ever be malicious alteration online legal documents but there are less there yeah I mean going forward is really important I mean it's two things is the opinions going forward and maybe a reasonable archive probably ought to be numbered and that's just the start because even if you got paragraph numbers you still need the local rules it's saying and you are allowed to in your brief cite my opinions using paragraph numbers and document numbers and that's we'll see take a while to get there you know having the courts know their opinions is a good start it makes it a little easier and many courts that do that do allow you to cite by document numbers instead of federal reporter citations so it's a start yeah I mean what's the experience of self-publication for us there's a lot of this stuff almost inevitably going to sort itself out yeah but document numbers aren't necessarily unique case identifiers I understand like the Ohio system where they have a they've assigned a number but Peter Martin has documented cases of states taking control of their archive and the number of paragraphs and then allowing you know the interneutral citation to occur and so that's a path and I think that's an area where pointing to the ABA and a AALL is is probably the safest thing to do and it's nice to depend on that existing consensus and authority that's out there for something I had a bunch of Egyptian programming to whom I was trying to explain the problem of unique identifiers for Supreme Court and trying to deal with the complexities of the identifiers system along with the system of government it was far more than I could and of the two they understood the system of government much more quickly I have to say so we are short on time at this point anything further before I abandon ship here and turn it over to Barbara I think more of us in a second okay I'm going to plug in