 I heard this morning a sort of pyramidal set of presentations starting at the most abstract level with the large-scale information policies of the United States. Moving in through with Dick Tanner's wonderful presentation, the history of access to legal materials in the United States, David Levy's salutary set of cautions about the kinds of things that would have to be dealt with, and finally Carl talking about the promise of law.gov. In the afternoon, we're dealing again with a set of perspectives that I think really add to our conversation. First, Jennifer Jenkins, who's the director of our Center for the Study of the Humane, is going to be talking about assertions of copyright over state laws, state statutes, state decisions. And for many people, this is the most shocking idea, the idea that states could actually claim that the law is copyrighted. This morning, Carl talked about nationalizing the law. Some people had the fear that this was nationalizing the law, making it publicly available. Of course, for it to be nationalized, you'd first have to assume that it was private to begin with. And some states actually make that assertion. Jennifer will tell you a little bit of the background that stretches over 100 years of the legal history of assertions dealing with assertions of copyright over the law. And then we're moving in. Are we going straight to David after that? No, and are we? To Erica. I have Erica scheduled to be after me. Do you want to change it around? Doesn't matter. I just have to be out of here at 3 o'clock. So that should still be fine. So then we'll be moving to Erica Wayne, who's been leading the initiatives to establish a national inventory of legal materials. Erica joins us from Stanford, Dean Levy's former law school. We're very, very glad to have her here. Thank you so much. And this, I think, deals with many of the issues that Carl was talking about, which is what actually counts as primary legal materials. And first of all, in order to figure out to gather everything together, we actually have to know what everything is, identifying that. And that's a fascinating project in its own right. And then we move to David Ferriero, our former colleague here at Duke, head of the Duke libraries, head of the MIT libraries, head of the New York Public Libraries, and now head of the country, I gather, at least. You realize there's only the United Nations after this, right? Vatican. Vatican. OK. Their archives have stuff you don't want to know about. I'm not talking about that. Oh, the whole thing. OK. So David, who has led a series of impressive efforts while National Archivist, is going to talk a little bit about what these whole sets of initiatives look like from the perspective of the archives of the United States. This goes a lot to some of the issues both of access, but also preservation, which we were discussing in the morning. Because after all, it's not enough to make things open. The question is, how will they remain open? How will they be indexed? Will they be findable? Will they be readable? Will the formats be the same, and so forth? And we also have a dear friend, a former student, I have to say. This is my only claim to fame, is that Andrew McLaughlin is one of my former students. The Deputy Chief Technology Officer of the United States describes himself as deputy chief nerd of the United States on his Twitter account. Former policy chief at Google, Andrew McLaughlin. Andrew is someone who's really working in the trenches to deal with the question of what would open government access to open governmental records mean in the United States and how could it be made to work. So he's really someone who's dealt with these issues day to day and dealt with the problems on the larger scale. Dean Levy provided the problems on the judicial scale. I think Andrew can give us a sense of the problems on the national scale. So with that, I'm going to turn it over to Jennifer Jenkins. Thank you very much. Thanks a lot, Jamie. Surely you have other claims to fame. Thanks, everybody, for coming. Thanks to all of the presenters this morning. Professor Boyle, Dean Levy, Dean Danner, Carl Malamud for fascinating presentations. Took us on a journey through history, and I'm going to take a glimpse back into history by starting with a quote from 1886 by a district court in Minnesota considering whether the law was copyrightable. The court wrote in eloquent 19th century language, it is a maxim of universal application that every man is presumed to know the law. And it would seem inherent that freedom of access to the laws should be co-extensive with the sweep of that maxim. Knowledge is the only just condition of obedience. The court then took us even further back into history to show that this tenet kind of goes without saying and explain the laws of Rome, ancient Rome, were written on tablets and posted that all might read and all were bound to obedience. So the question I want to ask today is in terms of public access to the law, are we less advanced in the age of the internet than in ancient Rome? My presentation will focus, as we said on what Carl earlier today called the jaw dropper, claims of state copyright over their laws, over everything from statutes and judicial opinions to regulations, ordinances, things like building codes and fire codes. In April 2008, a kerfuffle that Carl was involved in, the website Giustia, which posts legal materials free online, received a cease and desist letter from the state of Oregon claiming copyright ownership in the Oregon revised statutes. Carl was also posting these statutes on publicresource.org and the cease and desist letter said that the state will take any and all appropriate measures to compel your compliance with copyright law. Interesting. Is this just an Oregon thing? You know, they're measly rogue archivists, they're rogue state, they're out west. They're like, oh yeah, we have copyright in our statutes, but it's an anomaly, it's an outlier. No one else is going to do this. No, as Carl said earlier today, the current count is that eight states are claiming some sort of copyright over their statutes and 21 states, correct me if I'm wrong, are asserting some sort of copyright over their administrative codes. So this is not an anomaly and the question is, is there any legal basis for this? The US Copyright Act in section 105 explicitly excludes works of the US government, the federal government from copyright protection. So we know that federal statutes, judicial opinions and other materials are squarely in the public domain by statute. But the Copyright Act is silent about state law. That said, courts dating all the way back before the 1886 case that I quoted from have repeatedly held that the law, often in quotes the law, whether federal or state law is not copyrightable. And they've done so for two broad and different sets of reasons. The first is the fundamental tenet or precept that I began the talk with, that citizens who are presumed to know the law and obey the law need access to the law in the first place. Ignorance of the law is no excuse. And it would be perverse to impede public access to the very laws that govern its conduct. So citizens have a right of access to the law and there are both First Amendment and due process constitutional implications to this. And recently in 2002, the Fifth Circuit in a Texas case confirmed that this broad rule, this broad tenet applies to both federal and state law. The law is the law, whether it's coming out of the federal or state governments. In a case called Veek versus Southern Building Code Congress International. And I will quote from the case that trace the court's quote continuous understanding from 1830s to the present that the law, in quotes, whether articulated in judicial opinions or legislative acts or ordinances, is in the public domain and thus not amenable to copyright protection. And then it added, there is no reason to believe that state or local laws are copyrightable. This applies across the board. So that's the first set of arguments. The public's supposed to obey the law, of course they need access to the laws. How can you copyright it, restrict access, put it off limits to the public? The second set of arguments about why the law is not subject to copyright protection is more narrow, more precise, and has to do with the limits of property rights per the doctrine's internal to copyright law. So as Jamie said this morning, copyright only covers original expression. Doesn't cover facts, the height of Mount Everest, the amount, staggering current amount of the budget deficit. It doesn't cover ideas. Freedom is good, bacon tastes delicious, and improves anything that you put it on, in my opinion. But beyond that, copyright's domain is original expression. The manifestation of you, or me, that is revealed by your expressive choices, whether it's my brushstrokes, my arpeggios, my haikus about bacon. And yes, there are a lot of haikus about bacon on the internet. I've learned this recently after finding a site called the World Bacon Society. But I digress, that's original, that's original expression. The Supreme Court has emphasized that the original expression requirement is no mere accident of the copyright statutory scheme. It is the essence of copyright and a constitutional requirement. So let's think about the law. Is the law original expression? Are we saying, wow, that guy who drafted the tax code must have a beautiful, tortured, and complicated soul? It seems like the opposite of an expressive choice, right? If anyone is speaking, it's the state, and as courts have noticed, by extension, the citizen's us, the people who want access to our laws. And as Hobbes told us, it's not a counsel, the law, but a command. So seen this way, the law would seem to be at the very heart of what is not copyrightable. It's not original, it's not expressive, reflecting the expressive choices of an individual author. And arguably, it's either a fact or an idea or both. And that brings us to the merger doctrine that Professor Warl talked about earlier, where there is only one or very limited ways to express an idea or a fact, and ideas and facts are not copyrightable. Idea and expression become inseparable. They're merged. And so this is called the merger doctrine within copyright law. And you can't copyright that particular expression because it would circumvent the prohibition on owning facts or ideas that would effectively confer copyright over unprotectable subject matter. So the V court, again, held as much. The V court was talking in that case about building codes. And it said, it should be obvious that for copyright purposes, laws are effectively facts. The US Constitution is a fact. The federal tax code is a fact. Its regulations are fact. The Texas Uniform Commercial Code is a fact. They are unique, unalterable expressions of the idea that constitute local law. So the idea is that, yes, you're expressing the idea of a fact of law, but there's only one way to do it. And another court said, it would be very difficult. An individual wishing to publish the text of a law cannot develop his own unique version and still consider it an authoritative copy, right? You cannot express in an act of law in any other way. Expression and the fact the idea of the law merge. So one copyright tenant, law is at the heart of unoriginal expression and not copyrightable. In many cases, the states that are asserting copyright over their laws realize this. And in fact, in the Oregon case, the cease and desist that are realizing that the law, the text of the Oregon revised statutes themselves, itself was in the public domain, did not claim ownership of the underlying text, but of Oregon's particular arrangement, organization, compilation of the law. Because putting section numbers in numerical ascending orders is very original. And sorting aspects of a statute by the subject matter that they pertain to, also a very original act. I'll read from the cease and desist letter. So this is interesting, this is a different claim, right? Okay, the law is not copyrightable, but what we're claiming copyright in is our original unique compilation and arrangement of all these different aspects of the law into our statutory code. And so the committee said, although the committee does not claim a copyright in the text of our law itself, we do claim a copyright in the arrangement and subject matter compilation, the prefatory and explanatory notes, the lead lines and numbering, I wasn't joking about the numbering for each statutory section, the tables, index and annotations and other such incidents as are the work product of the committee in the compilation and publication of original law. So the question is now we're not talking about the underlying text, what we're talking about things like section numbers, titles, headings, page numbers, are these original enough to be subject to copyright protection? We have Supreme Court guidance on this for all of you who are copyright scholars and law students. There was a Supreme Court case in 1991 called Feist versus Rural Telephone Service, which involved a telephone directory that put the names of its subscribers in alphabetical order. And the claim was that, well this took a lot of time and effort and a lot of work, so surely we should have copyright protection for this telephone directory and the Supreme Court said no. The sine qua non of copyright protection is original expression that says there has to be originality, some spark, some minimal degree of creativity. And the court held that merely putting things in alphabetical order doesn't fly, right? Not sufficiently creative. So the question is does putting parts of a statute under subject headings in ascending numerical order sounds kind of similar to alphabetization. It sounds sufficiently trivial, sufficiently lacking in creativity as to not bear it copyright protection. And in fact, in an analogous case, the Second Circuit in one of the many cases involving West publishing held that the following acts with regard to judicial opinions were not original. Rearranging information from the cases that specified the parties, the court and the date of the decision, adding certain information about who the lawyers were, adding subsequent procedural developments such as amendments and denials of rehearing, editing, peril and alternative citations to delete obscure citations and that sort of thing. Quoting from the court said West's choices, the choices I just mentioned on selection and arrangement can reasonably be viewed as obvious, typical and lacking even minimal creativity. And I would argue that in many cases, these state claims of copyrightable selection, coordination and arrangement would also fail the requirements for copyright protection under that standard. To quote the late Professor Patterson, they see ministerial rather than discretionary or expressive. And the merger doctrine I discussed applies here too in terms of selection, coordination and arrangement, how many ways are there to organize and arrange the law, right? Like quick, Carl, make a version of the Oregon statutes without the official structure and section numbers, get creative, invert everything, rearrange them, put section one over here in section, right? It's not gonna work. So again, the expression there emerges with the underlying facts. And on top of that, if we think of the goal of copyright itself, copyright law exists by constitutional and premature to provide private incentives to create and distribute works in order to benefit the public, right? And are these copyright incentives necessary in the case of laws? Judges aren't gonna write their opinions and legislatures aren't gonna draft statutes without the copyright incentive. No, the incentives aren't there. What about the other part of the equation? The purpose of copyright law is to incentivize creation of works so that the public can access and enjoy them. That part is perverted as well, right? The incentives aren't necessary, but they're actually impeding the public access. So if you look back to the goals of copyright itself, applying copyright to these particular statutes and judicial state statutes and ordinances, regulations doesn't seem to further the goals of copyright at all. So those are two sets of arguments that you find often conflated in the cases. One about the broad right of the public to access the laws that govern its behavior. And the second about the rules within copyright itself, copyright only extends to original creative expression. It doesn't cover facts and ideas. So we find those in the cases. There is a movement largely led by Carl Nothers towards opening state law to finding these copyright claims. Will it happen? I was talking to Carl yesterday. We have a lot of states and they obviously have different stances on this. A centralized solution would be infinitely preferable than going state to state, probably. What would that look like? Well, there actually was a proposal from Senator Barney Frank in 1992 that would have put state laws within the ambit of the Copyright Act. I mentioned section 105 would have covered, exempted both federal and state works from copyright protection. So that would be a very direct and sweeping way to solve this problem. However, with the way the legislature works and everything they have on their plate, I think it's probably unlikely to pass. Another solution is a judicial solution. Say in the Oregon case, there was actually a lawsuit and it went all the way up to the Supreme Court and the Supreme Court said, okay, we've reviewed this and know these statutes or the selection coordination arrangement, the statutes themselves, the text absolutely not copyrightable. Given the amount of happenstance, effort, years, vagaries of the court system that it would take together, that's probably unlikely as well, although we see some cases that could be very helpful. And so what we're seeing is this public movement, right? This law.gov movement to try to promote states to actually open up their laws, to not claim copyright over their laws and what would that help to achieve? Well, one of the things is what this workshop is about, public access and public access, not just to the federal laws that we were talking about. Dean Lebe was talking about judicial opinions, federal statutes, but state laws, a lot of important things. Carl was telling me yesterday that he's put all of California's building codes, I think, online and he gets fan mail from plumbers, right? People who are doing renovation projects that are like, thank you so much for putting this stuff on there. I mean, imagine how useful that is if you're a plumber or doing it. You're just, you know, trying to fix up your house to actually find a place where you can find the building codes online, right? This having access to these state and local laws are just as important. In fact, I don't know why I find this quote funny, but the quote was trying to come up with things that the public would do with these ordinances. And it said, citizens may produce copies of the law for many purposes to educate their neighborhood association simply to amuse themselves, which I guess, yeah, there's not a lot going on. Indicating their dissatisfaction with their complexity, et cetera. But yes, of course, having these laws publicly accessible would be very useful. The other thing, and there's a concrete example from the Oregon scenario that I talked about, is innovation. Carl said earlier that the law.gov project is about justice, democracy and innovation. Well, the good news is in the Oregon case, two months later in June of 2008 after much fighting and discussion, Oregon decided not to enforce the copyright it was claiming in its statutes. But they had also noted said that, by the way, we put our laws online anyway. What's the big deal? We're satisfying the public access requirements. So Carl ran the statutes through the system and he found the following things. I believe he found over 500,000 HTML errors. He found that they didn't meet the Section 508 accessibility requirements. They had no metadata. They were using a robot's text file to keep search engines from indexing. So yeah, they were online, but they weren't terribly useful. What happens after Oregon says it's not gonna enforce this copyrights? I don't know if we have any law students here. Ah, we do. Enterprising second year law student at Lewis and Clark Law School set up his own website. Actually, I don't know if it's a guy or a girl. Do you? A guy. A guy. Okay. His own website. And it's user-friendly. And it's browsable. And it's annotated. And Dean Levy was saying earlier today, we don't hire web designers in the court to maintain our websites. Well, a lot of state budgets aren't being dedicated towards web design either, right? So why not open up the law and let the enterprising law student or a librarian or a legal information specialist or web designer or anyone actually create a useful website, an attractive website, a website that people can actually browse easily, build a better website. So first, the public access, the example of the plumbers, is two, innovation, in addition to justice and democracy. Three, some solution to the copyright claims, defeating the legal uncertainty would be a particular benefit to risk-averse entities, obviously. There are those who are going to put things online and risk the lawsuit, but then entities like librarians, universities who might otherwise be engaging and extremely valuable, do notwithstanding, because we have Dick and Keith, and we do good things, Kevin. But universities and libraries that would otherwise be authenticating and preserving things wouldn't have that barrier to doing so, wouldn't feel that they were gonna be subjected to a lawsuit. And finally, it might actually be good for the states themselves. I understand the state's concerns, their concern is that they actually make a bit of money from licensing their purported copyrights to the private publishers. You know, books, those books, those legal books, can cost several hundred dollars. And so they feel that there is a monetary incentive to these copyright claims. And also they, in some cases, say that it helps them to be the authenticator, the verifier of their laws. But are these claims really good for states? I saw something that said state employees themselves, you know, the people who often have very limited access to the legal resources that they need. It would be very useful to them to have things freely available online. And of course, the state's goal is to offer their citizens access to law. And so this would seem to be a shared goal, the goal of the Law Dog Up project and the goal of the states. So I've been talking about copyright during this presentation, and that's only one part of the problem. And fixing the copyright issue would only be one part of the solution. Because saying something is not copyrightable doesn't mean I actually have a right of practical access to it. So the FISD court said that alphabetically arranged telephone directories are not copyrightable. That doesn't mean that UNC has to put its directory online, right? That's only one part of the picture. And so I would argue in addition to really achieve the goals of democracy, justice, innovation that we are looking for, we need not only a change in the law, which is certainly a very useful first step, but also a change in attitude that would promote real meaningful access to the law. We need materials to be up to date, verified, authenticated, and released in a way and through a website that actually provides meaningful access to citizens, for example, no digital rights management. You go to some sites and you, sure you can read the law, but can you copy it? No, say you're in Arizona and you think maybe it'd be useful to translate some of their new laws into Spanish so that the people who are being affected by them might be able to know that their rights are being taken from them. Can you do that? No, right? And so unencumbered access, unencumbered by things like digital rights management schemes, unencumbered by the provisions of click wrap contracts, for example, just click here and agree to these terms of use before you go and use the law. So both a resolution of the copyright issue and an effort, a change in attitude and a joint effort between states and grassroots efforts towards meaningful access would be a step in the right direction and this workshop is a step in the right direction. Thanks. Questions for Jennifer? Okay, we're going now to Erica Wayne. Erica comes to us from Stanford where she is deputy director of the Wall Library and a lecturer in law. She has written on these issues of access to legal materials. Also, I think, excoriated the U.S. News and World Report ranking system. Oh my God! It's not by my design. It's not by your design and teaches classes, including advanced legal writing and women's legal history. But it is on the subject of the inventory of legal materials that she will be talking to us. Thank you, Erica. So if I'm being difficult here, can I take this anymore? Can I go to a table? Sure. The headline yesterday, what was it in the times that we met the enemy and it is PowerPoint? I'm going to subject it to that. Oh, now you see my family pictures, my nightmare scenario. It's so funny! That is horrible! I was joking about this with you, Donna. Yes. Hold on one second. That is hysterical. Here we go. Sorry, now you have to see that. I hope I have the right flash drive. It's really funny. Erica's nowhere near as bad as what I saw once, the gentleman finished his presentation and then went on to do his email on the screen, not realizing that the next person was not using PowerPoint. It was a very affectionate email with a number of remarkably inventive nicknames. Now I'm starting to copy the picture into the hard drive. Oh, there. No, that's not me. No, that's not me. You want to go? Funny about my family pictures. We went to U70 last week. That's what we were looking at. Oh, my tax return. This is getting funnier. This is really funny. I grabbed it from the, I thought I grabbed mine. I think I grabbed my husband. That's funny. So we also want to deal with the privacy application. Yes, yes. And this is all on the record I call. I make Duke and Stanford proud all at once. Anyway, so I'm going to talk to you a little bit about the national inventory of primary legal materials. I did like the subtitles that the group dubbed the other. So I said this is the view from the law library instead of the view from the government otherwise. So in brief, let me get started. In a recent Golden Gate University Law Review Chief, California Chief Justice Ronald George wrote about historic reforms to the California court system and how these have enhanced access to justice and the like. However, there are problems. And Chief Justice George explained, courts in California currently operate more than 70 different case management systems with about 130 variations. The systems do not connect with one another and do not provide information across court and county jurisdictions. And he writes, we cannot afford to operate in an electronic tower of Babel. I thought that was really fitting. Even though Chief Justice George was only talking about California court's case management systems, a tower of Babel frustration exists for anyone attempting to do legal research today. Many of the legal research materials that we consider primary aren't freely available. What is free often carries a warning that it can't be relied upon or is not official. For every state, there are different vendor relationships when it comes to publishing the codes and the assertion of copyright on the material. The concept behind law.gov might help reduce some of this confusion. So what is the national inventory? For law.gov to work, we really need to create a national inventory of all the primary legal materials and then some. The inventory will be a packing list of sorts that describes details and catalogs where one can find the laws of our federal and state system. And not just the materials we consider primary. That's why a primary retails plus. We might wanna note the availability of those items that are created as part of the process from the briefs and filings of attorneys to congressional testimony and the like. For purposes of this inventory, I've actually used a definition of primary authority from the Legal Research Dictionary by Elise Fox and I think she put it well. It is the law itself, authority that issues from one of the branches of federal, state or local government as part of its function or that issues from the Constitution. So back in January, we held the first law.gov workshop back at Stanford. There's Carl. That's Roberta Morris. What you don't see is Anurag Churna from Google and Jonathan Zittrain on the other side. And it was a really well attended event and we had lots of law librarians in the group. In fact, we were really lucky to have from the Northern Association of Law Librarians, Northern California Association of Law Libraries, NOCAL. We had both the current president, the incoming president and past president among other folks. So there were lots of librarians there. And this topic caused a lot of interest and some of the biggest questions I just wanna share them with you because I think it's informative. What should we include in this inventory? That's the first question. What would we want to include? What type of materials like scope, content, format, price? What form should it take? Should it be a wiki? Should it be something fancy? How do we note potential copyright issues? People are always afraid of kind of saying the wrong thing. How do we note that? What about IP issues regarding briefs and filings? Which I think Professor Boyle alluded to earlier. How do we organize the efforts to create the inventory? Is it gonna be something done chapter by chapter on a national level? What about legislative efforts? Are we gonna get involved with that front as well? And what about the vendors? Do we bring them into this process, as Carl had mentioned? And how do we build upon the work that Dean Danner talked about, that the American Association of Law Libraries has written on the authentication process and so on and so forth? Are we gonna be adding to this or really just building upon it? So at that very day, there was such excitement generated that an informal task force was developed on the spot. And we decided that our mission would be to create a prototype of the primary legal materials inventory, but for California alone. And our thought was that if the inventory idea was to really take hold, it would need to be developed and populated across the states with lots of volunteers. But we had to take the first stab at it and do a California model. And since California is a really big state, we thought this could be a really rich example. And then other groups could follow us. So after a few phone calls, low tech phone calls, we created a Google mailing group and anyone can join here, I can give you the information at the end. And we decided to go for simple. We decided that if we spent a lot of time developing the greatest platform ever for this, it would take too long. And Carl has a timeframe on this too, so we wanted to kind of support that model as well. No one wanted to become Drupal experts in our group. No one wanted to learn something new. So we wanted to go with something simple. So we opted for a Google spreadsheet and I'll try and open that up in a second. Likewise, we decided that we could have gone the way of just using Word and just sharing our notes occasionally, but we did want to do something that was shared. And the nice thing about Google spreadsheets that I'm not plugging in in particular, and it's not because they're formal Google employees in the midst. But they are down the road from us, but we did think it was nice because Google is very easy for people to access. It's nothing scary, it's very easy to share, very easy to maintain access. And also the tracking changes was really nice and it continues to be a very nice feature. It's very easy for folks, and this is kind of a blank template if you want to look at it. It's very easy for folks to add content in it directly if they want to manually edit it, or we have a form, let me open up the next screen, that comes, you know, it's very easy to create and so folks can go onto this form and add what they'd like. And I guess with all, I should say, with all things created with forms and spreadsheets, there is always that modification improvement you want to make after you start it. I guess hindsight and spreadsheets don't mix. And we have, for example, like we actually should have clustered cities by their counties, we didn't. We will do this going forward, we will modify it, but it's unfortunate we didn't think that way at the initial point. Note to future groups. We also didn't include a category for permanent public access. And we plan to add that going forward, but it's something we just, we're thinking it's not enough of an issue in California. And likewise, one of the members of our working group commented, we can skip an authentication category because this is the best quote of the library mix. Nothing in the state of California is authentic. Nothing. Nothing. So that was classic. And there are also categories we created that even now a few months later, we're realizing aren't that useful. And so if the next version comes along, we would just cancel certain categories. But there are some very important categories and we are filling up some certain categories, especially dealing with copyright assertion, disclaimers, official status, and price information. And I should note, sometimes it's really noting that there is a price because it sometimes is hard to acquire that data. And we suspect that these are gonna still be important fields throughout the states, not just in California. And at this point, I wanted to share kind of a fun little story. I co-teach advanced legal research at our school and occasionally we'll bring some of our real librarian concerns into the students, maybe to see if they're awake, but just to kind of gauge there what they're thinking and one good example is Paul, who co-teaches the class with me, my director, will bring in very skinny supplements to texts and bring them into the student state. Do you know how much this costs? Trying to see if they have any sense of how expensive things are, trying to share as outrage at what publishers you're doing to libraries. And usually we'll get a very polite, ah, that's horrible. And I'm like, blah, blah, blah, back to Facebook or whatever. And that's it. But we decided to give our students a lot of homework project just last week, actually. And we asked the students to each, because we have almost 50 students, to each take a state code and look at them in paper and online form and focus on just a few factors, on copyright language, disclaimers, and official status. And they actually, I'm not kidding, I think they really enjoyed it. We also were easy on the grading too, probably. But they were really engaged in the conversation and incredibly surprised by what they found. They thought it was important and they thought it was something that they never really get to talk about in law school. So I'm glad we kind of opened their eyes to that. And maybe for another day I can share if there's time later I can tell you what they thought were the most interesting states were Montana and Vermont. But that's another topic, maybe, with what we saw. But speaking of kind of notable trends, let me share a little bit of some of what we're seeing right now in California. So of the almost 540 municipalities and counties in California, from what we can tell nearly 80% of them have outsourced their codes to just four commercial publishers. And that number might actually be higher as we go through this because some of the cities make it a little hard to figure out who produced the site. But that's what we're seeing so far. Likewise, and these are kind of interesting, almost all of these are online free. However, and however is what always kills you. The almost 40% of them claim to be unofficial and have disclaimers. Almost 50% have copyright assertions and there may be more. We think those numbers are gonna go higher as we look deeper. Some of the tricky part, and this gets to also working in a large group with different volunteers, is defining what's unofficial. Some people feel uncomfortable saying something isn't official unless it says it is official. I know this is kind of slicing hairs but a lot of our volunteers would rather say not sure. So we will probably find that number goes up as we go back through what they did. And likewise, equally interesting, the paper versions of many of these codes which are deemed to be official are published by the same group that publishes the online version and for a fee. The price information on these things can be really tough because you usually have to call the publisher to get the price. It's not something that's readily available. So here's an example. I know it's tiny, so I'm gonna just show you. This is the Laguna Beach Municipal Code and it's produced by Quality Code Publishing. I don't know if you see there insignia there. They produce about 90 of the state's municipal codes and most of them look, not the content, but the format looks almost identical. And they all have the same disclaimer and I'll read it to you as I looked at all 90 of them this weekend. The electronic version, and it's the tiny print at the top, the electronic version is provided for information only. Well, that's good. And should not be considered the official version of the code. Please consult the official printed version before citing provisions of this code. If inconsistencies exist between this electronic version and the official code and or the underlying ordinances, the official legislation will be considered definitive. Okay, so I said, well, gee, I wanna find out what's definitive. So I called the city of Laguna Beach just to find out how much it costs to purchase it and got a lovely person on the other end of the phone and she's like, oh, that's a good question. And I was like, do I have the right number? And she's like, oh, yeah, yeah. I think it's about $175, but I'm gonna have to get back to you. And I said, oh, why? And I think this was the quote, I haven't sold one of those and who knows when? And she's like, it's online. I was like, well, but do you sell them? She's like, yeah, but I'm gonna have to get back to you and find out. So that was not the only example, so more of the same. So here's closer to my home San Jose, the California Code of Ordnance is produced by American Legal Publishing and they do about 43 codes in the state of California. Some of them look a little different, but they all have the same disclaimer and notable they all have a copyright assertion at the bottom. This is copyright 2010, American Legal Publishing. I don't know if you can see that all rights reserved. Let me read you the disclaimer because I think that's also really good too. The code of ordinances and or any other documents that appear on this site may not reflect the most current legislation adopted by the municipality. So it's great, it's not even perhaps current. American Legal Publishing provides these documents for informational purposes only. There comes that only for information. Documents should not be relied upon as the definitive authority for local legislation. Additionally, the formatting of the posted documents may vary from the official copy and the official printed copy of a code of ordinance should be consulted prior to action being taken. So don't waste your time with this basically. If you did wanna buy the official one for San Jose, it was very easy to get the price. It's about $300 plus 158 a year to keep it current. One of our volunteers actually called American Legal Publishing to ask about the copyright notice. Wondering if it was something at the behest of the city or if it was something by American Legal and they said, have to get back to you. We still haven't heard. But this also got a few of our volunteers talking about this one because this disclaimer that you see here and in most of the municipal codes that I've shown you, you don't see disclaimers like this in printed books. I mean, any librarian here knows that mistakes happen in books all the time. You'll get those replacement pages or those self-adhesive pages to stick in, all the librarians not their heads, that you stick in the books. But you don't see disclaimers like this in the printed material. But it keeps coming up on these websites again and again and again. And it just, it just makes you go, hmm. And the hits just keep coming because this one's really good. This is the office of the Attorney General of California and we were on this site looking for the opinions of the Attorney General. And this is, I think, one of the best ones. Due to the dynamic nature of the internet. I love that. Resources that are free and publicly available one day may require a fee or restrict access the next. That's good, that's good. And I love that they also cannot guarantee that it will be error-free. So I guess due to the dynamic nature of print publishing, if you wanna buy this in official form it's $400. So that's tough, that's really tough. And there's more, there we go. Let's see, I'm only gonna show you a few. We're not going through all like 700 things we've looked at so far. This is what Carla mentioned but I think it's worth looking at again from the Courts of California. This is the disclaimer page. The first of the two disclaimer pages you get if you wanna look at the searchable opinions archive from 1850 to the present of California appellate and Supreme Court decisions. It should be noted that the State of California has a contract with Lexis, Matthew Bender, to produce the official reports. And even though the archive is free online you have to read through this agreement and then when you click on the continue button you click through to another license agreement you have to select that you have to agree to that's over 2,500 words long, quite detailed. But let's share some of this because I think this is good too. There is no charge for using this and they're saying there is only copyright, no copyright in the text of the decisions but only in, let's see, the head notes and summaries are subject to copyrights. They're not in the archive but the best is probably the last line. The official reports page is primarily intended to provide effective public access to all of California's presidential appellate decisions. It is not, this is classic, it is not intended to function as an alternative to commercial computer-based services and products for comprehensive legal research. You mean Lexis, yes, it produces this. And I swear to you, if you go through the license agreement on the next screen and I don't feel like doing it because I think if you click through you give up the right to your first born. I mean it is very specific what it talks about like you can't reproduce it, you can't re-engineer it, you can't do anything with it. I think you're just supposed to look at it in awe. And one of our LR students when we were talking about this last week made a comment, would this be malpractice for me to use that? And I think that's a really interesting question. One we will not answer today but a really interesting one. And I think this is the last of our humdingers. This is actually on the court side as well. The California courts produce civil jury instructions. This is called khaki in California and this I'm sharing with you as an example of a confusing copyright illustration. The khaki on the California website it actually claims to be official online and it is identical to the paper version. Lexis is the official publisher, Matthew Bender Lexis. But look at this copyright assertion. We were joking that's like the Christmas turkey because every little part of it has been carved up. I mean you, the assertion is blah, blah, blah, blah, but not to this, this and this. However, Matthew Bender has everything but. So it's incredibly detailed and what's notable too in the paper official version it looks the same. And for the unofficial version done by West it's basically cut and paste this and just strike out Matthew Bender and put Westlaw in and it's the same copyright. And this is a very difficult example for our inventory because we just had like a check box that said yes. And this really doesn't really fit neatly into yes or no because it was pretty complicated. And there's probably gonna be more like this that we see as we go along. We noted that there is no assertion by Matthew Bender to life expectancy tables and we figured that's good because ours will be shorter after reading this. And so that is a challenge which brings me to the challenges we have with the inventory. So the first and foremost is probably finding volunteers and keeping them interested. We're actually pretty lucky we have about 20 volunteers in our working group and we send out annoying notices on email getting people to sign up. We also, I hit up a lot of folks who came to the first workshop saying if you are interested enough to come, will you please help? And I should also acknowledge that some of the volunteers actually work at Stanford Law Library. No peer pressure there, but we're happy to have them. And it is hard to keep people motivated because it is a bit of work. And getting fresh blood is good but I think some things that help are actually free stuff and swag. And I'm so glad Carl has like these little badges and magnets. So this is really good. We're hoping we get more of that. That'll be really fun. It helps. Little free stuff goes a long way. Just for what it's worth, we also joked about having virtual happy hours while we're doing this. But there's a call to have a real one and given that we were all pretty slap happy about some of this stuff. We're thinking we'll do it in Lemon Grove, California. Anyone from Lemon Grove? If you look at their municipal code site, they claim to have the greatest climate on earth. So that would be a nice place for a drink. But that gets me to municipal data. Filling in all the blanks is really tricky. There's a lot of content that we need to look at and it's stuff that even though many of us have been law librarians for a long time and things we haven't seen. These are not things that we use every day. So it does take a bit of time to get through all these towns and counties. Consistency is also tricky because there are many different ways of defining some of these things and I was mentioning version control and tracking changes is really helpful on Google spreadsheets because every once in a while I'll see a certain volunteer will make changes and there's always the shy volunteer and the very bold volunteer and I'll know who they are. So we're like, oh, so-and-so edited. Better go back and check because I know that certain people are just a little cautious and I just need to keep an eye on that. The pricing data is really tricky because as Daner knows this and other folks here know this, vendors have different relationships with so many different entities. So you might have a different pricing plan, different discounts and we've learned that it's quite challenging to get exact prices. If there is such a thing, I don't even know if there is. Maybe we need Ken Spengales who does a pricing guide for legal information to help us kind of fight this down but we're really just pretty happy to get information that there is a price and I'll just share this as a funny kind of hiccup. So one of our volunteers, one of our best volunteers works for a court and when she signed up, she was like, you know, I recently ordered materials that you're looking at for the court and I know how much they were. I did the invoices. They're all on my computer. I said, great, can you tell us how much they are? She's like, well, according to rule of court, blah, blah, blah, you need to file an official request. I was like, okay, so I'll file an official request for these prices, which was really funny because it went right to her desktop and then she opens the request and she shoots me a personal email saying, I see your request. As soon as I approve it, I'll put them all into the inventory and so they took like five extra steps but it's all up there now, so it's good but I think these are one of the things if you are going to work on this project you should just know about. Looking forward, maintaining the content is gonna be tough because even though Carl is probably gonna be done hopefully this summer, I think this inventory, if we start it, we should keep it fresh and constantly updated because otherwise it's really just going to be a relic and getting other chapters to do similar work. I think today, AALL government relations folks are actually going to send out a notice with documentation to seven states about getting their inventory started, which is awesome. I'm so excited, that's great and I think it's fitting because when Jonathan's a train spoke at our event, he said that we were at the, not the birth of the idea because Carl had that some time long ago on a deck somewhere or something with a drink but this is the cradle of an idea and so if we really do want this to thrive, we need to take good care of it and see that it evolves. Speaking of, how many North Carolina librarians do I have in this room right now? Raise your hands. Actually, better, can you guys stand up and take a bow? Because we love librarians. Okay, so you know who you are. Do you have scratch paper or a laptop? Yes, yes, nods. Okay, so I'm gonna give you exactly one minute right now and I want you to write down the title of a primary source in North Carolina. I see Dean Danner's doing it and I want you to write this, if he's doing it, you should do it too. But yeah, all the cool kids are jot down the publisher name if you know it. Is it online? Is it official? Is it free? Anything you know. Just spend a moment. I've eaten up like 20 of the seconds, so just no pressure. No wrong answers either, so just go for it. I'm not really watching the clock, so just look, it's like Jeopardy. You can collaborate. This is the beginning, yeah. Okay, I think I'm gonna call time. Did you all write something down? Some of you did, yes? Donna did. Congratulations, the North Carolina Inventory has been born. You started it just now, that's the best part. No, it is that easy, that's all I want you to know. It is easy, don't get intimidated, you just have to get started and if any of you want blank templates or forms, see me, I am happy to supply them and I just want to close with saying that I think as librarians we have a natural claim to this, so we should own it, document it and keep it alive, so that's all. And my information if you want to contact me later. Thanks. Try to use 70 or attack me. Yeah, I have one, Erica. Often when it looks at pricing information, the pricing is for a single copy for yourself. And I'm wondering are you asking about licensing if somebody wished to make a copy of a municipal code and say put it online for everybody to use? That's a good question, we're trying to get that but the tricky thing is if we're not able to even get the cities to tell us how much their paper code is, getting a license answer is going to be even trickier. So I think this is going to take a lot more nudging. So here's a suggestion, maybe you could get a shill like the company Justia to send in letters to some of these code publishers and inquire about commercial licensing and what the prices might be. And this would be very useful on a national basis. One of the things that we're hoping to demonstrate with the national inventory is that if Stanford for example wanted to put a common set of all the municipal codes of California online, reformat them and valid HTML, re-index them, how much would that cost in order to get the licenses to do that? And you can't do it for all 540 but if we had a dozen price points it might be a useful thing. No, and that's good but the tricky thing as I was saying is like we're still having such a hard time getting the cities to tell us how much their codes are which just should tell you something that they're not even able to tell you how much their official printed version is. And I think that lesson is important for North Carolina and the other states in that you're not gonna get a definitive listing of everything but painting the picture of what the situation is like, where might there be issues? Are they in the judiciary, the executive at the state or at the county level is gonna be very helpful I think for yourself as librarians but also on a national basis for painting a picture of access to the law. So this is more a question, a more common than a question Derek but it seems like it would be really wonderful to write up as a short piece for, I'm trying to think, the green bag. I didn't know, some periodical that might be read by people in the judicial conference that might be your horror stories. Because horror stories are actually, I mean, studying the history of the environmental movement, there was high theory but there were also quilting of horror stories. And it's the, here is the canal that is on fire. Here is the place where you can't see each other across the canyon, right? And these are just too funny to be kept to your PowerPoint besides which the next time you probably are gonna show your tax return. So it's higher risk for you. Speaking of horror stories. So I just think it would be, I think that'd be really powerful because when David Levy today was offering really thoughtful and I thought, good correctives on the problems that had to be gone through but then his reaction to the idea that states are claiming copyright over their codes. Even just that idea is that's just offensive and that's just over the idea that they're claiming it let alone that they're making these kinds of assertions. And so I think this is really the wedge that makes people realize the importance that I think Carl grasped that long time ago. Thanks. Thank you so much for doing it. Thank you. And now we turn to David Ferriero who I already introduced but I just want to say welcome back. I believe it was Tom Wolf was from UNC said you can't come home again and you said that we Duke can come home again. So thank you. It's really nice to be back on campus. This is, I've told everyone, my eight years here were the best eight years of my life but I've only been at the archives for five months so stay tuned. So it's really great to be able to give you the view from the archives after five months. So the National Archives and Records Administration has a specific role in its clear and simple we're the nation's record keeper. We safeguard and preserve the records of our federal government and make them easily accessible to our citizens so that they can use them and learn from them. The records that come to us for accession into our permanent holdings represent only about two to 3% of all those created by the federal departments and agencies. And I can tell you it is one billion pages a day that come to us. And those are just the two to 3% that are of permanent records. But they're the most important records. Once upon a time and not so long ago these records were all on paper and we provided access to them wherever they were located among the National Archives facilities around the country. 14 regional archives, 17 federal records centers, 13 presidential libraries, the National Personnel Records Center in St. Louis and our facilities in Washington and College Park, Maryland. Over the years, many thousands, millions of people have come to these locations for genealogy information on the forms of ship records, immigration roles or civil war pension files. Millions more contact our St. Louis facility for their military records to qualify for government benefits. Others consult the records of Congress. We hold in Washington to enrich our understanding of representative government and sell others to visit our presidential libraries to do research about one of the 13 presidential administrations for which we hold records. The National Archives has been and continues to be a leader in the online movement for greater access to primary legal materials. Already we provide a large amount of these documents to our customers on the web. Since 1994, NARA's Office of the Federal Register and the Government Printing Office have provided primary legal materials to the public, the public and private laws of the United States, the United States statutes at large, the daily federal register and the Code of Federal Regulations. In 2008, the federal register began placing documents appearing on public inspection online, thereby providing the general public greater access to the documents that impact their daily lives. The Federal Register and GPO partnership now offers bulk downloads of federal register and Code of Federal Regulations XML files to the general public via data.gov. Bulk downloads of this information are being used by the Legal Information Institute at Cornell in the Center for Information Technology Policy at Princeton in new and innovative ways to help citizens play a greater role in the rulemaking process. FedThread.org has been experimenting with user annotation in terms of making comments directly beside the text of federal register documents, which could then be submitted as comments to the official site.govpolts.us is also on the forefront of the transparency movement by making the federal register searchable, more accessible, and easier for the general public to participate in government. It's not only the private sector that benefits from bulk downloads. Providing this important legal information has created new opportunities for government, allowing us to develop new ways to display this information online. Executive branch agencies and departments send their important records to the archives according to record schedules. They are then available unless they fall under exemptions or exclusions under the Freedom of Information Act. But many executive branch agencies put policies, rulings, and opinions immediately on their websites long before such records are scheduled to come to NARA. It all forms a significant part of administrative law. Eventually, the most important of these documents will go into the Electronic Records Archive, or ERA, that we are building to hold all the federal government's electronic records. This includes both those that are born digital or traditional records that are digitized. The idea is to make these records accessible far into the future, free from dependency on any specific hardware or software. These records then will be accessible to the public at any time from anywhere in the world. The first phase of ERA ran from 2005 to 2008 and involves electronic records from four federal agencies, the Patent and Trademark Office, the National Nuclear Security Administration, the Bureau of Labor Statistics, and the Naval Oceanographic Office. These agencies now use ERA. Later this year, the number of federal agencies that use ERA will be expanded from most four to about 30. At the end of the George W. Bush Administration in January 2009, the ERA took in 77 terabytes of data from the executive office. That's about 320 million searchable objects. These records from the Bush Administration will be available in accordance with the laws governing presidential records. Eventually it will be mandatory for all federal agencies to move the permanent records to the ERA, which is on schedule to be fully operational in 2012. The prime contractor on this project is Lockheed Martin and during the four and a half days while the government was shut down in Washington, I had an opportunity to read the Lockheed contract for the first time. All 3,000 pages of it to see exactly what we had signed on for. This is a massive project, the most expensive, most visible, most complex technology project that I certainly have ever been involved in. So it's very important that I understand what it is that we signed on for to ensure that we deliver what's supposed to be delivered. Everything I've mentioned so far fits in well with our open government plan, which was created to comply with the president's open government directive. The president's directive calls for the creation of a culture of transparency, participation, and collaboration in and among federal agencies. The goal being to transform the way the government does business and the way people interact with the government. The thinking behind open government is the essence of the work we do every day, rooted in the belief that citizens have the right to see, examine, and learn from the records of their government. Our open government plan goes further. It strengthens the culture of open government within the National Archives. It strengthens transparency in the workplace. It provides leadership and services to enable the federal government to meet the challenges of the next century. And important to our discussion today, to develop web and data services to meet our 21st century needs. We will continue to contribute data to data.gov. So far we have posted the federal register from 2000 to date, the code of federal regulations from 2007 through 2009, and descriptions from our archival research catalog dating back from 2002. These descriptions now cover about 60%, 65% of all the holdings of the National Archives. We plan to leverage the internet to make our records more easily available. Our website, search capabilities, digitization strategies, and our use of social media to engage the public must be able to meet these needs. One of the things that will be different will be the federal register. We're going to relaunch it as a daily web newspaper for the 21st century. The new federal register will guide its readers to the articles and topics that are most popular and relevant to their lives. It will have individual sections for money, environment, world, science and technology, business and industry, and health and public welfare. This new approach will also highlight proposed and final rules that have significant impact on the U.S. economy or raise important regulatory policy issues. Also, the new federal register will post a calendar of events constantly updated to list public meetings all over the country, including those that offer webcasts and remote call-in options. The calendar will also track the opening and closing of comment periods and the dates of rules going into effect. Users with an interest in a particular agency can easily follow each day's documents as well as the most popular documents issued in the past. Statistics and visualizations will track agency activity over time. Other innovative tools will appear in the new federal register, one of them being a regulatory timetable that pinpoints where a regulatory action stands in the official process and links to previous proposed rules and related notices. For those new to rulemaking, the site will also offer tutorials, articles from academic contributors and access to government document librarians. The new federal register will go beyond just reading about government rules. It will help people participate in the government, one of the major goals of the Open Government Initiative. Each document that asks for public comments will feature a highly visible button for submitting direct comments directly to the official agency site. If a reader wants to share news and comment opportunities with friends or interest groups, the document will include a feature for sending email and posting to social networking sites. At the same time, we'll be redesigning our public website to maximize public participation as well as to develop streamlined search capabilities. We intend for our entire website as well as access to our holdings online to be user-focused, a user-focused community experience. Furthermore, we intend to explore ways to develop our current catalog into a social catalog that allows our online users to contribute information to descriptions of our records. And those of you who have caught my blog, AOTUS, will see an interesting conversation going on about my concept of the citizen archivist and a fair amount of pushback from the professional archivist about giving away the store to the general public. I've always, my experience has always been that we learn more about our collections when we talk to the people who are using the collections and those people walk out the door with all this wonderful content. So this is a way of capturing some of that content. So stay tuned. Our Open Government Plan is posted on our website and I encourage you to take a look at it. Your comments and suggestions are most welcome. And if you have a comment for me, you can check out my blog and I'd be happy to hear from you about the law.gov project or any archives issue. We are poised to open up government more for its citizens and that of course means having a dialogue with those citizens. So I'd be happy to talk more about that. Thank you. Questions for David? I have one. Can you tell us a little bit about things that were surprises to you? Things that were surprisingly easy or surprisingly hard in your attempts to open things up and any reflections on what we can learn from those surprises, on those things that worked easily and were pushing downhill low-hanging fruit as things that were extremely difficult and seemed almost impossible to change the way things were done? A lot of it I would say has to do with the culture of government and I'm just reflecting on my own agency. And it was similar to my experience when I walked into the New York Public Library an attitude that we've always done it this way. Why would we ask anyone? Why would we look elsewhere for solutions to problems? Those kinds of things. A lot of staff have been in place for a very long time and a level of insularity I would say that was pervasive. I've put a lot of stock in the president's open government directive because this was an opportunity rather than a tops-down approach, a bottoms-up approach to rethink how we go about doing our business to make not only agency but our work much more transparent, much more collaborative and much more participative. So it allowed me to identify a layer of staff within the organization who'd never had an opportunity to shine and to charge them with coming up with the open government plan for the agency and the culture has already begun to shift. I discovered, two months ago, I discovered the social media working group of the National Archives, which had been meeting in secret because it was not authorized. It had not been authorized. The private social media. I caught wind of this and walked into their meeting and sat down with 25 people who are the most creative folks in the National Archives so far that I've met with the most creative ideas that have been unleashed, basically. So that was, I was not prepared for the depth of the conservative bureaucracy. That doesn't mean that we've solved all those problems, but the message is clear now after five and a half months that things will be different in the Archives. So just a follow-up, if I could, on that. That's one of the things that we sort of study here at the center of this idea, sort of cultural agoraphobia that we're always see very accurately the dangers of opening things up and oversee them and we underestimate the positive things. We, you know, the idea of the internet, the idea of Wikipedia, the idea of open source software. These are all things just like they sound ludicrous if you imagine yourself, you know, 15, 17 years ago. And then they come and the question is, but do we learn from them? Do we generalize or do we think? Well, that's special. That's a weird, special thing. Obviously, when you talk about the idea of a citizen archivist, you are challenging a priesthood in both the good and bad senses of that word, right? But we actually, we take as a serious obligation going back through history, the idea of preserving the archive and you want to let it in the hoards. What, often people are not convinced by arguments but by concrete examples. What concrete examples do you try and point them to to say, look, we can do this and it won't be as bad as you think? We had a great example last Friday afternoon. We had a meeting. I meet once a quarter with the research group, the folks who come to the archives and do research. So they, anyone who wants to come to this open meeting comes and I met and I presented the citizen archivist concept to get some reaction from them. They were excited. I mean, they were to beside themselves thinking about things that they had discovered in the collection and one guy stood up and he had just this week found a George Washington letter and a revolutionary war diary in pension files. We didn't know they were there. We had no idea. We've got 10 billion things in the archives. Literally 10 billion things. We're never going to know everything that we have in the collection. So he's going to be our first citizen archivist. I mean, he's going to be featured on my blog next week. So sharing those kinds of discoveries and there were three or four staff members in this and when they heard this, they were. So telling stories is I think the best way. Other questions? The other thing that you asked me about challenges. The other thing that I wasn't prepared for was the level of classified information that I'm dealing with. Another mandate from the White House at the end of December was the creation of the National Declassification Center. By the end of 2013, I have to have reclassified or reviewed for declassification more than 400 million pages of content. There's another space door. We have in the federal government, there are something like 254 agencies. Within those agencies, there are more than 2,400 different separate classification guides in operation. So we've got this massive problem of varying degrees of standards and implementation of classification rules that we really have to streamline and work through. One of the issues with law.gov is obviously going forward and getting current materials, but one of the other issues is scanning the back file for primary legal materials. And by that, I mean Supreme Court briefs, congressional hearings, back issues of the CFR. Do you have any idea how big that problem is and if not, might the National Archives be able to help us determine the scope of that across the three branches of the federal government? We need to talk. Okay. That would be useful information because when we go to Congress and say here is the scope of your problem. Great, thanks. Happy to talk. And some of these things, Supreme Court briefs, for example, are available from commercial vendors, but they're very, very expensive. And presumably in the archives, you have copies of Supreme Court briefs, I hope. Jonathan? Hi, I'm Jonathan Wiener from Duke Law School and I'm a former OSTP staffer. But my question may be more general than just for you, but I thought you might have particular insights. The current, this month's issue of Harvard Magazine has on its cover something like the end of the library, and it's about the internet putting pressure on university libraries, even as a gigantic library system as Harvard's. Bob Darden? Pardon? Bob Darden? I'm not sure. An article by Bob Darden? He did not write it. He quoted it. So my question, and if someone has, if you've already discussed this earlier today, I thought of a lot of questions. I wasn't here earlier today, so this is fresh. But just, what do you think is the, what do you think of that, of the changing role of the library, including non-profit university libraries, municipal libraries, but also what's the role of the government's archives and Library of Congress and so forth in responding to the potential pressure on the non-governmental or non-federal libraries? I think it's having spent my entire life in the research library environment and having watched the role that technology plays throughout my career. It's clear to me that the future of libraries is heavily dependent upon technology, that at some point, I won't live to see it, at some point all the content will be available electronically. We are, you already know this, those of you who are in the classroom, that if it isn't an electronic form your students aren't finding it, they're not using it, there's no such thing as paper to most undergraduates anyway. So the more we can do as a community of getting as much content into their hands, eyes, minds as possible, is the role that we should be playing. As the archivist of the United States, my goal is to get all 10 billion of my items in digital form. However, I can do that. Working with Carl, working with, I'm about to have some conversations with Google, ways that we can get all of that content accessible. I can't speak for the Library of Congress, that's a whole different situation. They have a very different mission than I do. I'm now focused on my records. Can I, one follow up on that? One of the things we were talking about this morning was the danger of deals when government needs to bring in a private vendor to make accessible materials and is very focused on one kind of access. I need to put it up online and then makes a deal which... I'm not happy with that at all and Carl knows this. We have some arrangements with some commercial vendors like ancestry.com and family history for genealogical material. So they have come in and scanned massive amounts of material from our collection. And they have the archives before my time signed a standard digitization contract. This is fairly standard now in the business of five-year lock on the content. This is, these are government records we're talking about. I'm not happy with this. Well, if that's everything we're gonna let David go. He is going straight from here to the plane. So I think an extra round of applause for him, Ted. Just before Andrew comes up here I do wanna say that one of the nice surprises of arriving in New York was working with Beth Novick from Andrew's shop. Beth is a lawyer from the NYU Law Faculty and she came and spent a Sunday afternoon with me at the New York Public Library to talk about her work, my work and how we might work together. And it's been one of the real joys of being in Washington is working with you guys. Thank you so much. Andrew, come to the podium. So we've had one view from government and now another view. And do you have like a big disclaimer first about how, you know, you're- The expansive claims you're about to make about the future of American Intellectual Property Policy or not, do not bind the administration? It's funny, I don't think of you as the kind of person who pushes disclaimers. I've actually- It's a standard practice. I'm collecting them. Well, so let me say that what I do have to say and what the lawyers always tell me to say is I'm not speaking on behalf of the administration or the federal government, I'm here in my personal capacity so take my views as my own. So my favorite book dedication of all time which is well-known to most of you in this room is Larry Lessig's in code. His book code, he dedicates the book to Charlie Ness and dedication says to Charlie whose every idea is crazy for one year. And I have to say that Carl has sort of taken that and compressed the time frame down to a matter of months at this point. So what I viewed initially as something of a windmill tilting exercise around law.gov has taken off and has gathered steam largely through the workshops that this is one of but also through the advocacy that you've been doing in front of the courts of appeals, in front of the executive branch agencies and so forth, it's really cool to see. It's been really kind of amazing. I've described numerous times internally in the White House a set of projects that we're doing as the Carl.gov initiatives. So it's all things that Carl has cooked up that we've now kind of embraced and are running somewhat sheepishly with because we wish we had thought of them first. Jamie, by the way, sort of qualifies under Professor Boyle, sorry, qualifies in that category too because I was indeed one of his students. The book that you wrote while you were teaching me was Shaman's Software and Spleen's and because you were my professor, I read it and I thought this is the most esoteric and weird thing I've read, it made no sense to me at the time but I've come back to read it a couple of other times and it is one of those books that was way ahead of its time in terms of understanding the way in which intellectual property law is literally the regulatory scheme for an information world and why taking those laws very seriously is essential to getting it right. Anyway, so let me just move on from the wild praise everybody to talk about sort of the intersection of law.gov and the open gov work that we've been doing. So on January 21st, the day after we took office President Obama issued as his first memorandum the open government memorandum. So this is four square corners kind of formal document. It's obviously available online and it mapped out the work plan for the open government initiative for the first kind of year and change and it was built around transparency, participation and collaboration as the kind of three organizing principles for the work and it said that we're gonna spend basically the rest of 2009 at which point working on the initiative and getting it right and understanding what we're gonna do and then we'll issue a directive. So the directive came in December and that's what David Ferrier was just referring to the directive says every federal agency has to cook up an open government plan and it's gotta cover the following topics and then they're all gonna be made public and what we're actually doing right now was we're in the process of grading them. So we're gonna be sort of green, yellow, red grading the different agencies on a bunch of dimensions that we asked them to address and we'll keep working our way through this hopefully kind of highlighting and celebrating the successes and shaming the laggards and working our way towards genuinely bottom up in a certain sense from the agency side anyway bottom up development of open government practices. The broad project of the first year in terms of open data was really about saying like let's figure out some data that we can make public and then let's get that data to be public. The longer term project is about culture change. So this is also something David just referenced but it is fundamentally the revolution that we're trying to pull off. The great thing about Moore's law over the last 30 years is that it's made computing power so cheap and networking in terms of the price per bit is cheap and is getting cheaper all the time. So you can have this vast sea of free stuff that we all take advantage of online every day. You can take the technology that makes that possible and apply it in a governmental context to make transparency, participatory, rule making and policy making activities, collaboration across the previously isolated and siloed units of government. All of this becomes possible and in fact, economically compelling, not just doable. So that's kind of the theory. The next year is going to be about trying to instantiate culture change in more concrete ways. The work that David's been doing at the National Archives is kind of emblematic of this. He mentioned the federal register. One of my favorite examples of what happens when you take the federal register and start publishing it in XML rather than locked up in either PDFs or the other kinds of formats that it had been published in is that it enables third-party developers to make applications. One of the ones that he mentioned, FedThread, allows you to do the following cool thing. You can type in your zip code. You can type in search terms and then anytime something that gets published that matches that, you get a text message or an email or shows up in your RSS reader. That's really cool and it, by the way, puts out of business a number of companies that have made as their business model the job of reading the federal register and alerting their clients every time something relevant to them turns up. Now suddenly, they're gonna have to do something that actually creates value rather than just rides on scarcity which no longer needs to exist and, in fact, does not exist. So that's the kind of awesome service that you can provide when you do something as simple as taking text and making them available in machine-readable standardized formats. So we've gone from the memo to the directive to the plans. There's a lot of work that needs to be done on data.gov in order to really make it sing. We need to build out developer tools. We need to build out APIs. We need to catalog and index a lot of the mashups so that people will feel recognized and rewarded as part of the community. We need to wrestle with a lot of the standardization of data and metadata. Enrich and enhance the metadata we've already got available. One of the things that I wanna highlight though is attention and it's at least tangentially, I think important in the law.gov context, broadly speaking and that is the following. One of the things that surprised me or was new to me anyway when I came to Washington was that the culture of government while often derided as conservative and risk-averse and closed and insular and all of that also has a very, very important virtue which is that there are many people in the government who take very seriously the quality of the data that the government puts out. They work their fingers to the bone to ensure that the quality of the data that they're putting out is excellent. And the culture that produces that is not something I think to be jettisoned lightly. Many of the labor statistics, economic statistics and frankly weather statistics, you name the data set. If there are people out in the economy relying on it, you want your government to be obsessively focused with ensuring that that data is very high quality. And I think one of the tensions then is we're pushing agencies to move faster to publish their data sooner, get it up more rapidly, reduce the lag time between creation and publication and we have to figure out how to strike the right balance. Now there are some ways to do this. For example, when we publish GDP statistics, we do it in three waves. You publish the initial estimate, then you publish a revised estimate, then you publish the final number three months apart. And economists in Wall Street and people relying on those numbers know that you apply a discount factor to the initial numbers and maybe a smaller discount factor to the next set of numbers and then you rely on the numbers when they come out. So we know how to, it is possible to publish tentative data. But one of the interesting experiences, one that was I think very tense or sort of pivotal for the Open Gov movement was, and this is, I can talk about this as an observer because it was an independent board that did this, but there was a board established with Recovery Act funds, the stimulus bill in January 2009 called the Recovery Board. Recovery Board's job is to oversee the spending of stimulus money. And one of the things they did was set up a website called recovery.gov where all of the data that is being reported up from recipients of stimulus dollars gets pulled together and published. So anyway, there was a moment last fall when the Recovery Board dumped this huge massive pile of data onto recovery.gov. Nice website, they actually did a nice job of it. I think we learned some lessons on the technical side about it. But what was worrisome was that as soon as the data went up, from tens of thousands of recipients, many, many different people reporting data, so of course there were errors in the data. It wasn't possible to scrub the data, certainly if we wanted to get it published in a timely way rather than waiting a year and then publishing it farther down the road. And so I won't name names, but certain people went on the warpath and attacked the Recovery Board and the whole recovery effort for the fact that there were zip codes that didn't exist, congressional districts that didn't exist and so forth. So we were worried about this because we didn't want the lesson of this to be, be transparent and publish your data, hand ammunition to your enemies to come and pillory you for all of the terrible and evil things that you're doing. So part of the culture change then is not simply getting government people to think about public data as the default and to be rewarded an incentive for publication rather than hoarding. Part of the culture change is also more broadly in the community to get people to understand that data can be published and it's not perfect. And we will collectively work together to figure out what the errors are and report them and then fix them. And from the perspective of the Recovery Board or the publisher of data, the right reaction to these kinds of complaints is thank you. Thanks for pointing out the errors, let's now go and fix them and collectively we'll make this data set better. So anyway, that's sort of a kind of an interesting, to me kind of an interesting lesson about the nature of the culture change that we need. It's not just inside governments, it's within the community of users, broader community of people that are interacting with the data. There is a metadata issue here which is how can you signal the level of confidence that you have in the data even if it's not 100% to be nice to have metadata tags and fields that can help people express whether data is tentative, subject to revision and so forth. That's something I think we can fix. One debate that we've been involved in that I wanna highlight is a debate around public access to taxpayer funded research. And I can't express any views about this because we have an ongoing, we had a public notice of inquiry at the Office of Science and Technology Policy. The comment deadline is now finished but we haven't published the output from that. I can't tell you anything about what we think about it but what it addresses is the question of when the taxpayers are through a research agency like the National Science Foundation, National Institute of Health and so forth, paying for research, should the papers that result from that research be free and publicly available? The leading agency in this area is the National Institutes for Health which has a policy of 12 months. So they say 12 months after your publication of a paper in a peer review journal, it must be made free and publicly available on the internet. And so the question that the White House posed was what do you think? Should this be expanded? Should it be addressed to different fields? Should it apply to the data that underlies the papers? Should it apply to different disciplines in different ways? Some disciplines might have publication schedules and expectations that are different from other disciplines. Economics might be different from bioinformatics or whatever and finally, how do you do this in a way which preserves the viability of publishers as providers of peer review and the functions that they provide that are sort of valuable? What's the right length of time? Anyway, these were all questions that were flagged. And anyway, it's interesting because in a lot of ways I think that addresses a lot of what you think about that is similar to what you think about the Law.gov project, which is to say, are there ways in which running law related databases on a user fee basis is better than law related database on a free and public resource basis? The same thing is sort of true. If you believe that the journals and publishers and providers of peer review are providing a valuable service, then how do you configure the incentives to enable them to keep doing that but still vindicate the objective of public access to information over the long term? And I think that's one of the things at least on the federal side that we've got to work through is we've got the PACER database which is a fee supported database. And there is an argument about whether it should be made free and publicly available to whoever wants it without having to pay money or should continue to be supported by the lawyers and law firms and other people who primarily access it today. So I think that's all I'll say for right now. I'm happy to take questions and happy to also touch on other things if you think there are other things that are worth touching on, Carl or Jamie. So can I start with the same question that I asked David which is coming into the job? Things that you found to be easier than you had expected, things where there was resistance and if so what the sources of those resistance are and sort of strategies for overcoming resistance to these initiatives? So interestingly, part of my answer is the same as David's which is that I'm endlessly surprised at the frequency with which sort of classified national security kinds of barriers are erected to the things that we wanna do. And it's just an orientation for the federal government. They take seriously and I think rightly so the obligation to keep people safe. And so the idea that bad things shouldn't happen on your watches is very common. And we have many, many different agencies that have some intersection with a classified world. And that means they've got a unit that does classified stuff and that protects data. So anyway, I've been surprised at that. I thought that here I am as a civilian going into a civilian role, I wouldn't interact with that very much, but we do. So... Classified intellectual property treaties, for example. For example, that's exactly right. In the trade negotiation world, broadly speaking, that stuff does not happen in public. Diplomatic negotiations don't typically happen in public. And I will say the US supported the publication of the ACTA text which went public I think last week. So that is a step forward on the transparency front. It took some doing, but it was a step forward. But it makes people nervous too, right? Because there is a pretty compelling case to be made for a lot of diplomatic stuff to happen not in public. And figuring out how to persuade people that you can do so in this case without jeopardizing your ability to do what you can be public and open in this case without jeopardizing all of these other important things you need to do over here, that's a real, that's been a challenge. Some things have been easier though, and I will say that in every single agency that we've had to deal with, there is some small cadre of really bright, talented, dedicated people, actually many of them, but many people that are like that, but a small cadre of people who really have kind of a fervor around open government and changing the culture of the agency and opening it up. I haven't found any agency yet, were there, that is bereft of that kind of person, and that makes the job much easier. To follow up, if I could, one of the difficulties that we found with Science Commons is getting people to think about multiple visions of access. So a lot of people understood the idea of open access for the Mark I human eyeball to a printed version or a digital version of a scientific paper. Scientists should be able to read, so they got that vision of access. But the idea of access that would be machine readable, and as you mentioned, integrated with data, the idea of access of a type where we don't even know the kinds of things that will come from the access because we haven't thought them up yet, which is entirely the point of why we need the access, that is a much harder kind of case to make because you're talking about, you are talking about things that have not yet been invented as being one of the things that are benefits from there. One way to have a kind of confidence is that, is to come from a culture, say a Silicon Valley culture, where you've seen that happen again and again, and you start to think, you know, if I walk out onto the crowd, they will actually, you know, bear me up. There is a reason to believe this will happen. Obviously that doesn't always happen. David gave the lovely example of the guy who finds the George Washington letter in the bankruptcy records, and maybe that makes people rethink the idea that opening the archives to citizen access makes sense. Have you found ways of explaining to people why access means more than just letting some people read for less cost? And if so, how did you use examples and concrete solutions in order to make that point real to them? So that's a great question. So, so anecdote, example and fantasy, right? These are kind of the levels at which we've been trying to make these arguments. So the anecdotes are all about a real live, you know, examples where, you know, you publish data, it gets mashed up in unexpected ways, and it becomes very, you know, useful. So for example, I mean, the federal register example I gave before is, you know, one, Tim Berners-Lee tells this great story about a bicycle crash data that in part of what was the centerpiece of his case to the British cabinet. So Gordon Brown had this, you know, the labor party of this political scandal about expenses. And so suddenly they felt of a strong compelling political reason to show that they could do something on openness and transparency in government, which coincided beautifully with Tim Berners-Lee's, you know, kind of urgent raw data now, movement that he was trying to push on the British. So they actually brought him in and he got his chance to pitch the cabinet on sort of open, on what became data.gov.uk, but it's sort of an open data approach. And so anyway, I'll tell you the long story that he's got a TED talk, which is worth looking up online and watching, but he tells the story about how the police had pulled back a database that listed bicycle accidents in London. And he showed how it took exactly 48 hours from the publication of bicycle crash locations for someone to put up an app that would show you the least crash-likely route to get from anywhere in London to anywhere else in London on a bicycle. So you want, you know, according to time of day, but it literally took 48 hours because all you need to do is have the data, have the routes, you know, have an online map. They did it on open street map. And anyway, it's like, you know, it's awesome. So now all of a sudden, you're going to have fewer people dying from bike accidents because the police put this data up online and other people were able to mash it up and do it very quickly. The Fed thread, which the Princeton, Ed Felton's group, the Princeton put up on the backs of the Federal Register took 24 hours. You know, like XML goes up. 24 hours later, I'm entering my zip code and getting Federal Register notices anytime something comes up. It's awesome. You know, it's totally, it's totally great. So anyway, that's the level of anecdote. The examples are sort of, what I mean by that is kind of celebrating the people who do the right things. And, you know, one of the best examples here has been not so much about public data, but about simply the value of digitized data. And that's in the context of electronic health records or the smart grid would be another one that we could talk about here. So electronic health records, when you get, when everybody in this country, which will happen by 2015, has an electronic health record in a standardized format. That is private, secure, but also easily exchangeable from healthcare provider to healthcare provider and easy to, into which it'll be easy to import labs data, pharmacy data, all this other stuff. So you're going to have this sort of record. Once that exists, using fairly rigorous anonymization and de-identification techniques, you're going to end up with this huge pile of data points. And we will be able to do really many, many things, but three things that I think are worth highlighting. One is you can improve the patient outcomes dramatically because, you know, the patient's data is easily accessible. You can, you know, figure out what the right diagnosis is based, based on knowing a lot of things about them. Second thing though, which is where the data is important, is in aggregate, you can do something called comparative effectiveness research, which is, you know, to say for a patient that has my characteristics with my symptoms, what's the best outcome I can hope for and how do I get there? And the way you figure that out is by doing statistical analysis of all the patients that had your characteristics, your kinds of symptoms, figure out where they got to and what got them there. What are the different outcomes? So you can test drugs this way out in the real world as opposed to highly sterilized clinical trial settings, which are important but limited in what they can teach you, and it can tell you quite a bit. Finally, the big thing that you'd like to be able to do is move the whole system from pay for procedure to pay for outcome. The only way that you can fairly price outcomes is if you've got massive piles of data, but if you could ever pull that off, it would be awesome for doctors, for example, because then their economic incentives are precisely aligned with the patients, which is do exactly what you need to do to get them to the best possible outcome, but don't do anymore. And if you do, you get paid this much and you may have an incentive to outfit them with a wireless heart monitor or whatever. Anyway, it's gonna be really great, but it's only gonna be done if there's this massive pile of data that you can do the analysis on. There's a similar story around the smart grid, which is basically digital home energy use data that will flow out across the wires that's coming. Actually, the utility company for North Carolina is one of the most interesting and adventurous and sort of forward thinking in this area. But anyway, homes will be outfitted with smart meters, they'll generate data, you'll get these big databases of data and then you can figure out how to reduce energy consumption yourself across communities, very useful. So anyway, those are examples of how to sort of do it. And then the final part is kind of fantasy, which is outlining what might be possible. Just by way of analogy, one of the biggest struggles that we're engaged in right now on a policy level is trying to free up more wireless spectrum for data. So moving it off of inefficient old uses for wireless data. And we are confronting exactly the problem you outlined, which is that we are absolutely confident that the technologies to take advantage of this spectrum much more effectively will exist because we've seen them in the lab, but we're asking people to kind of take on faith that they're really gonna flourish and develop in a way which in Silicon Valley seems totally predictable. Like of course we're gonna pack 100 gigabit per second connections into air, we're gonna be, of course, we're gonna be able to do that. But we're asking people to take it on faith. And in the context of sort of the data problem generated that remains a problem, remains a problem to tell people, to convince holders of data in the federal system that their lives are gonna be easier, their jobs are gonna be, their agencies are gonna be more effective, they're gonna look better, because some third party that they haven't even met is gonna go off and do things with this data. It's not an easy, it's not an easy challenge. Thank you. Other questions? I got one. Jay? Sorry, Jay. No. Public health and national security are often seen as public goods where the government will invest in software development. Holding government accountable is also a public good. And I wonder if you see not just government making the data available, but also helping develop the software in an open source way that would help us understand the data. So this is a recognized problem, at least by us. So one of the things that we've sketched out for data.gov, for example, is we need to add a much more robust set of developer tools. So that's on the roadmap for this year and we've got new people coming in on detail to work on data.gov that will be prioritizing that. One of the sort of tensions around public data, I mean to add another one to the pile, is the tension between raw data and visualization. So we've been telling people, don't worry about visualizing. It's a natural inclination for people to think how they're gonna make the data look. But just get the data out there in raw, unvarnished form as data feeds. And then also worry about visualization, but just do it as a very separate activity from the data. Having said that though, for people to really hold the government accountable, they have to be able to see and grasp and comprehend and visualize the data. So that's another thing that we've got sort of on the roadmap is simple standard, not spin-proof, but spin-resistant visualization format, simple graphs. This is kind of Edward Tufty style visualizations, but simple, straightforward presentations of data that can be applied to any data set that exists within the data.gov universe. Beyond that, I would hope that agencies will develop and contribute analytics tools that go take you beyond visualization to do things like mashups and cross comparisons of data across multiple dimensions and that sort of thing. That's not something we currently have on our roadmap. I could imagine that we might, but I'm hoping that that'll be sort of development out in the agency world that we'll be able to build on or hijack into the data.gov universe. We're still kind of operating on year-to-year planning cycles for data.gov, and that's just not there yet. Coffee or a question? Yeah, so in terms of anecdotal poster children, the Federal Register is a great example. It went from ASCII tests and PDF to XML. The paywall went down, it's authenticated. In order to find other good examples and bad examples in the executive branch, where should we be looking? Or do you have ones off the top of your head? And we're talking primary legal materials here, so rules, regulations, codes. Well, so let me say this. I think there are a lot of bad examples by which I mean I think there are a lot of agencies that sit on rule sets and official publications that are not currently even in the pipeline for digitization and standardized machine-readable formats. That's sort of the bad news, and I can think of many of those. I mean almost every agency publishes manuals, guidelines, and rule books of various descriptions, particularly the ones with rule-making authority, that are not really even in the pipeline. They may be low-demand, they may be out of date, they may be things that they wanna keep offline for various reasons. In terms of good role models, I would say one that I'm kind of a fan of is Health and Human Services. So HHS produced what I think is a great open government plan, and it explicitly addresses this idea of clearing up the backlog of all of their official materials, all the official stuff, the rule sets, and everything else that are published. In fact, they mentioned Monaco was an inspiration for that. That is not coincidentally true. But that's one that we've been pointing to because that was a really awesome forward-thinking kind of plan, and that's one that I would sort of use as a role model. But I think of law.gov broadly, which is not just the primary legal materials that are things that carry the force of law, but more broadly, agencies do a lot of regulation, and a lot of that stuff happens in the form of soft things, like if you think about the DOJ, prosecutors guidelines. I mean, that's critically important to understand the way in which the law is being applied and is an example of the kind of thing that should be up online because of the critical nature of the information that it contains for people that are living under our laws. In my opinion, that counts to its primary legal material. There's a differentiation between authoritative legal materials and those are things with the force of law, but primary legal materials, to me, are things issued by governmental entities that might be used in interpreting the authoritative materials in a congressional hearing, I think is a great example of that. It's not the force of law, but you can't interpret a law without the congressional hearings that led to the enactment, and so those manuals and hearings and dockets, in fact, that led to rulemaking procedures all, to me, are within the scope of the law.gov effort. Maybe not as central as, let's say, a Supreme Court opinion, but to me, those are important in that governmental entity that issues the law needs to be able to point to those and make them available to the public. Yeah, I mean, if we collectively take is our starting point where Jennifer started us off, you know, namely, the problem of things that a citizen needs to be able to read to understand how the citizen is to behave or where the limits of freedom lie or however you wanna think about it, that material is important because it tells you what the agency is going to do, how they interpret the law. Anyway, so I agree with that. We have, that's something that we have a lot of work, I mean, there's a lot to be done in that area before we can sort of look in the mirror and feel good about where we're at. Okay, thank you. Let's thank Andrew. I'd suggest we just take a five minute break, feel free to go to the bathroom, get some water, whatever, and then we're gonna start our final panel, which I think will be relatively short, just talking about sort of upshots, deliverables, see where to go now. And then following that, we hope you'll join us for a reception. So let's take a five minute break and then reconvene at around three, three. Thanks.