 Today's webinar is the Peggy Project Takes Flight, and we have three presenters with us. First is Deborah Caldwell, who is the Diversity Resident Librarian at the University of North Carolina at Greensboro. Scott Matheson is Associate Law Librarian for Technical Services at the Lillian-Goldman Library at Yale Law School. And Robbie Fattel is the Government Information Librarian at the University of North Texas. So thank you very much for joining us. All right. Thanks, everybody. Before we get started, if you could answer in the chat for us, please, if you're familiar with it and whether or not you've attended a Peggy event before, and we'll just give that a couple of minutes. Okay. I noticed we got a – I'm not sure. So just to kind of – well, you'll hear more detail in a moment, but just sort of to let you know, we did a series of mini forums at a bunch of meetings over the past year. So you may have seen usually one of the three of us or one of the other project members sort of asking for your input or thoughts on these, on some of the preservation of electronic government information. It does seem like we've got some folks who are not too familiar. So we'll do a pretty broad overview, sounds like. Yes. So for those of you that aren't familiar with us, we are the Peggy Project. We are here to raise awareness about the preservation of electronic government information, and we thank Linda and the NCLA for offering us this forum to share more about what we are doing, what we have been doing, and what we hope to continue to do. We did, as we started to form ourselves as a group, set a mission statement. And so on your screen is the mission statement that we've developed for ourselves. We started as a two-year initiative to address national concerns. We're entering our second year of the project. So we may revise this mission statement as we move forward, but our hope is to raise concern about the preservation of electronic government information and what organizations can do to help preserve it or advocate for preservation, all of those types of things. And these are our partner institutions, just a little bit of background. I am at the University of North Texas, which some of you may know is pretty well versed in preservation of digital content. We do a lot of print conversions to digital format, as well as we participate in the end of term crawl. We also do other forms of harvesting and digital preservation. And so it was sort of a natural fit when Martin Helbert was our dean to engage in conversations about preserving electronic government information. And he attended a depository library council meeting in 2015 in which he was part of a panel discussing preservation of government information. And he was kind of frustrated that everybody kept talking about print. Because yes, print is important and print is fragile, but we don't even know the scope of electronic information out there. And there's not a lot being done to preserve it. And so in 2016, we hosted our first meeting in conjunction with the Coalition of Networked Information, CNI, in San Antonio. And we had about 50, maybe 40 people at that meeting and definitely saw that there was interest. There was some diversity in thought and expectation and understanding. And so we decided we needed to do more to raise awareness and to have conversations and to discover the greater issue. And so we had a second meeting that following December again as part of CNI. And at that point, the data rescue events were hot topics and happening. And awareness was being raised in the general public's mind about these issues. And so from that December meeting, we started to put together a steering committee. And so our steering committee comes from these institutions. Dr. Halbert has since moved from UNT to UNC Greensboro. And so he is now there. We've partnered with the CRL Center for Research Libraries as well as EDUCOPIA. EDUCOPIA has provided a lot of institutional support for us through the process, as well as hosting some of the publications that we've produced as part of the project. We did receive an IMLS grant, which we are eternally grateful for. And that is what has made it possible for us to host some of the meetings that Deborah's going to talk about in a minute across this last year. And so the IMLS grant has allowed us to host the meetings that Deborah will talk about in a minute. And these are the goals of our project. They have evolved from the beginning when we first set down to discuss things. But our main goal is to engage a variety of stakeholders in meaningful discussions around preservation of them, persistent access to electronic government information. And we did use the forums, the mini forums throughout this year, as well as a variety of other venues and opportunities to do just that. We're trying to identify alignment between stakeholders for future collaborative activities. So we have done a lot to talk with people that are heavily involved in the data rescue events, data refuge, and projects that are hoping to move those efforts forward as well. We are in constant communication with our friends in federal agencies to find out what they're doing and what we in the community could be doing to support their efforts. We want to access stakeholders and other engaged participants who are receptive for using the collective impact model. We approached the, and Scott will get more into this, the National Forum in December with the hope to apply collective impact to our efforts and realize that maybe we can do it on smaller scales and grow from there. And then we want to access the landscape of creation, dissemination and preservation of electronic government information or assess that, sorry. So we have created, we commissioned an environmental scan, which I think Deborah is talking about that too. So we have attempted to do that and we are constantly learning that the landscape is vast and assessing it is difficult. So our project activities. In the spring of 2017, Robbie and I went to DC and Maryland to gather more information about these issues. We spoke to representatives from NARA, GPO, and the Library of Congress, as well as a small agency called the Marine Mammal Commission. And these informal fact-finding interviews and the fact that we were able to do them in that manner in person and in short succession were really helpful in contextualizing certain core themes and contentious terms from a variety of perspectives. The information gathering trips allowed us to speak with the information producers specifically, which was particularly helpful. And after that, we then convened a series of mini forums and various conferences to speak with experts in a variety of fields, as I think Robbie mentioned a little about that above. This was a broad range of fields, historians, scientists, archivists, preservation specialists, and others. The attendees from these various stakeholder communities were selected on the basis of both their ability to represent the perspectives of their stakeholder groups as well as their willingness to actively participate in the National Forum. So we wanted to engage a variety of stakeholders in these meaningful conversations around preservation of and persistent access to electronic government information. And in doing that, we convened these mini forums, as well as speaking with the agencies listed there. And we also had two webinars, one of which was more presentational and a means of gathering information from our attendees and the other which included a variety of our partners to sort of present out on related efforts. At these mini forums, we asked these three questions. One, are there categories of electronic government information that you think have long-term value? Two, do you identify any categories? Do you think there's any potential risk that threaten long-term survival? And three, do you think that there are any useful strategies for better coordination? And if anyone has any answers to these questions, I'd love to hear them. If you could put them in chat or also you can fill out our official survey on our website, peggyproject.org. Using the information gathered in these interviews, we then committed to drafting an environmental scan. Initially, we had considered doing this part of the project ourselves, but over the course of our information gathering, projects included previously mentioned interviews and mini forums, as well as because of responses to our survey questions, we determined that the environmental scan document would then best be accomplished by contracting it out, which we then did to Sarah Lippincott through the Educopia Institute. The environmental scan, the purpose was, well, like most environmental scans, we wanted to get the lay of the land. So Sarah Lippincott did a really excellent job in the environmental scan together with our project partners and steering committee members, Sherry Laster and James Jacobs, on really documenting the existing state of the born digital federal government information and the process and procedures and potential gaps identified. We concluded last year's activities with a national forum, which convened strongly qualified group of experts and key stakeholders to engage with these issues involved with it. As Robbie mentioned, everything happened in December in D.C. This was also held last December in D.C. in conjunction with CNI. And Scott, we'll talk more in-depth about that. Indeed, we brought together lots of different stakeholders for the national forum. You can see on this slide the group picture of us all up there on the top. But we tried to bring together who we had seen reflected in the mini forums, but also to pick out anybody who we thought maybe had been underrepresented or who we hadn't been able to get in touch with, all while trying to limit it to a reasonable number of people that you could put into one room and we could all work together. So we ended up with about 60 people, including the steering committee. And that was folks from GPO, NARA, journalists, coders and hacktivists. We had scientists, archivists, academics, of course. We even included a couple of folks from outside the U.S. who had been through sort of similar issues. So our Canadian folks came and told us a little bit in some of the mini forums, but also then at this session about what they experienced when their depository program went very electronic and what some of their preservation issues were and how they dealt with those or are attempting to deal with them. So we had this large group of people and then structured the forum activities into several parts, which are based loosely around the idea of the collective impact model. These were parts that we moved through over the course of the two days to test where we had agreement or we could come to agreement among the various stakeholders. Again, sort of with an eye towards testing whether or not we could implement sort of a collective impact model or if there was just too much disparate interest in the group. But you'll see we sort of worked through defining what the actual challenge was. Could we agree on what the scope of government information was and what was our electronic or digital electronic government information and what might be at risk of loss? So we sort of worked through that and there was some general agreement there. There were I think everybody probably found at least one thing that somebody else mentioned and said, oh, yes, I haven't thought about that, but that's an important piece. So once we worked through that, we then sort of the very next step was talking about vocabulary and some places where there were different definitions in different communities of practice, these trouble terms that you see on the slide. And these were things, you know, preservation means something different to an archivist than it does to a librarian than it might to a technical data preservationist. And we were able to suss out some of those, a big long list of terms which you will see when the final report is published if you look, and then actually define work through sort of beginning to define or at least define the differences for, you know, maybe a dozen of them. But terms like preservation, terms like archive, these are things where different communities of practice and different professions don't always agree on what those terms mean. And developing a common vocabulary is really a key part of this collective impact model because in order to all work together we all have to be speaking the same language. You see Robbie's put the link there into the collective impact website, and this came from the Stanford Social Innovation Review, and we've got citations to it. And I think just about everything that we have published, but it's not unusual, and we certainly can talk more about it as she says. So after we talked about vocabulary, then we talked about how might we, once we all agree on what the problem is and how we're going to talk about how do we measure that progress. And this idea of measurements is really, we spent more time on it than I thought we would. There was a wide agreement that we sort of understanding what the denominator was, what the scope of the problem was, was difficult and not well understood right now. That sort of, do we know all of the electronic government information that's being produced that people are interested in? People think it may be of ongoing interest. Numerator in that, sort of how to measure what it is that we're doing, what it is that we're actually preserving. That was less, there was less agreement on that, and there was a lot more diversity, which actually was useful and helpful in our discussions as a group of what different communities were interested in. So this idea of measuring progress, what the measures are and how you might gather those measures was interesting. There were discussions of measuring certainly things that you might think of, like with end of term harvest, where you might think, oh, how many websites are there that we want to preserve? How many did we preserve? What's the size of those files that we preserved? How many pages did we preserve? That, while sort of a difficult technical challenge, there seemed to be general agreement on. Other measures of progress seemed were more intangible and would be more difficult to measure, things like public awareness, the amount of use that these materials get going forward. You know, we're preserving these things, but is anybody using them? How do we measure that? Some of these things may not be measurable, but it was a robust discussion about measures. And then, sort of to wrap it all up, once we had talked about these first three elements, we talked about, you know, we just charged each smaller group to come up with an idea of what might you do to move this agenda forward, this idea of preserving electronic government information for access? How might you move that forward? And we came up with several different. This is a representative sample of four, or we think it's representative. We came up with about eight potential pathways, maybe nine, just sort of roughly outlined ideas. One of which was that number one there, Friends of Federal Facts, to sort of create a community around not just the preservation of existing data, but continued data collection longitudinally over time. And this is one of the key things that came up out of the data rescue management movement was, okay, we've taken this snapshot. How do we make sure that, A, the program continues to collect data, and how do we then preserve that ongoing data? Because that's what, especially the scientists need over time, is longitudinal data. Another group came up with an idea of looking at the assessment of government preservation by domain or by discipline. So this would, collaboratively, we would work together to develop a framework for how you would document efforts. And then domain experts and folks who were particularly interested in a given domain might work through that framework to assess that area by subject or by academic discipline or by domain, as opposed to the way we tend to think about it as government information librarians. What have we done with the Department of Health and Human Services? What have we done with the Department of Education? What have we done with the Department of Labor? You might instead take a domain view of public health and say we're going to look across all of those agencies and assess what data is produced, what's available, what information is well preserved or not. That was a way for those domain experts who are not necessarily government information experts to contribute back. On number three, this idea of a network for the data rescue toolkit. A data rescue toolkit is a separate standalone project, but this idea of developing a network that could be a resource for that group as they begin to pilot a nomination tool. This is the tool where you can send in a URL. You may be familiar with it from end-of-term harvest, where if you participated in any data rescues, there was a nomination tool where you send in a URL. And I think the idea of this expert network would be that it would not only be a technical tool, but also would provide more context and information and sort of a state of urgency. Again, those domain experts, that broader stakeholder community could help with the technologists and the archivists. And then finally, as our last example, we talked about increasing transparency of federal records decision-making. The kerfuffle over the Department of Interior record schedule modifications happened about a month before the forum, so this was sort of at the top of some people's minds. And we are happy. I am happy to say that just last month or maybe a month and a half ago now, NARA did make some changes or proposed some changes to how those record schedules would be evaluated and published again in the future. So this idea of it is not necessarily that anything particularly good or bad is going on, but just letting people know how these systems work, which again sort of feeds back into all of these. You start to see that theme of engaging the stakeholders with the importance of this information and taking some, if not ownership over it, at least some advocacy for it, so that they can continue to do their work. The work that we do as government information producers and professionals helps and continues to be useful and helpful to the people who need to use the information. That's a good old patron service. That was sort of the overview of the forum, the national forum. And that's where we are now. Some of our next steps as the PEGI project are to put the finishing touches on the national forum report, and that's happening now, and write our final grant report, and then decide as a group sort of what our next steps as the PEGI group are, and deciding, you know, what funding to pursue or how to proceed with moving forward with the momentum that we have. One of the great things about all the mini forums and the national forum is that, and events like today, is that there's actually some sort of, seems like a critical mass of folks who are at least aware, if not fully engaged with this issue or this ongoing. I think we maybe should talk a little bit more about collective impact. Robby, do you want to do that, or if there are questions, now would be a great time to help us out by asking for clarification. Yes, so just to address collective impact briefly, and that was a good question about did we consider other models? We did not. Our partners at Ejikopia are pretty versed in collective impact model, and bring a lot to the table with regards to that mode of approaching community-based projects, and so we just kind of went with it by default. And it has been, if you're able to read the report at some point, it has been successful in developing grassroots solutions or collaborative solutions to really localized problems. And it is more specific to the nonprofit community, so those organizations that are more focused on identifying and solving community-based problems. I think one of the examples in the report deals with teen pregnancy rates. And so it is a different sort of approach for our type of project. We do see this issue as being a community-based issue and one that could bring community-based solutions to address concerns. And so that is in part why we thought of the collective impact model. We did learn, as we said down at the National Forum, that the lack of clarity of scope definitely posed problems to apply broadly the collective impact model. And so that is when we started to adjust our thinking that maybe if we did have more specific projects that there would probably be opportunity to apply the process to smaller projects. It's an interesting model for when there is a problem that is too big for any one person or group to solve. And this sort of came up sometimes in the mini-forms. I think it's, well, isn't NARA doing that? Or isn't GPO doing that? Or, you know, OLC is archiving all that stuff. We don't have to worry about it. And it's, to a certain extent, some of those statements are partially true. NARA is doing a lot. GPO is doing a lot and trying to do more. But it's a large project to really do this well that needs everyone to pitch in. And that's what I think we saw with the data rescue movement, that sort of this idea that people wanted to do something. Building in those sort of a framework where we all agree, if you want to do something, here's the right direction to pull is what this model is good at sort of designing for folks. It's not perfect, but it's a way to kind of develop that common agenda. And I think the thing with the collective impact model is there has to be a very specific attainable goal through which progress can be mapped and measured. And it's still a little slippery for us. So Ann, I hope that both clarifies and helps us fess up that we didn't investigate too carefully any other models of kind of moving separate groups together in the same direction. But this one seems to work well, as Robbie said, with community problems where there is a community. And then you saw in those proposed next steps forward this idea that some of them were actually about building community around this issue. Yeah. And if you have ideas for other similar models, I think we would be open to hearing too. And are y'all, so you mentioned at one point the survey, the three question survey that y'all have been doing. Are you still doing that? Is that ongoing? It's still available on the website. That's a good question. I think we're trying to figure that out. It will make it into the final report, sort of an overview of what people referred, you know, what people responded to with that. I will say that I know Robbie and Deborah did more, but the three of us did a fair number of these many forms. And we start to hear even from different communities, even like at AALL, the law librarians said some of the, and the legal information preservation alliance said some of the same things that the Society of American Archivists folks said. There were clearly differences, but there were also, you started to see some convergence on some of those fundamental questions. The survey was just a very informal way of gathering this information regarding the three questions. We did get a lot more and more substantive feedback and response through the things like the many forums or the webinar that we held where there was a lot of interactivity and people interacting with each other and with us and answering the questions in the chat. And you can see that first webinar that we did, it's on our website. And you can, you know, hear through the video or we learned the chat posted, but you can hear through the video where we're, you know, speaking aloud to the respondents and answering questions like anything. We got a lot of response that way. We have the survey up as a means of collecting that information, but I don't know if this answers your question, Linda, but the specific, like, the web survey, the web form, it's certainly not some kind of like, you know, really rigorous, like, it's very informal. And so we're leaving that up there in case there are other things that people want to continue to contribute to us as simply another method for people to provide feedback to us. But as far as the information that was used to inform the report and other parts of the project, I think more of that was gathered through other means. Great. Any other questions for our presenters? I have a question. Robby, when do you think we'll have the National Forum report out in case folks are looking for it? That's a great question. I believe we will probably have it done by the 1st of May area, don't you guys think? Well, now I do. I think that's reasonable. So, you know, when you start to think about finals time, go look at the PeggyProject.org website and look for that final link to the final report. I believe we're planning to put it on the edukopia site, just like the environmental scan. Oh, we've got another question. Right. So, I think and we similarly through our conversations with groups that did data rescue events or were involved in data rescue had similar concerns that there were limitations. There were certainly technical limitations of participants sometimes. And, yeah, I don't know that we figured out how to address that going forward. I will say that, you know, and you make the point that it needs the involvement of the data creator. And I think that is something that we're seeing GPO and to a certain extent LC address as well where they're sort of GPO has actually commissioned sort of concurrent with the work that we were doing a little before. A study which is on their website of how different agencies in the government produce the information that much of which should actually end up with GPO and whether it is ending up there or whether it's not and maybe why it's not. So, that research that GPO undertook is something that sort of addresses that, you know, getting at the data host piece of it and making sure that the people who are doing the work. The very short elevator summary of that study is that the people who are doing the work are no longer publication specialists. They're often now either data scientists or they are the web host or the IT department and they don't have the same publication mindset. And also, I want to give a shout out because Ann mentions, you know, that a lot of things were kind of, I ran the data rescue debt and group and we had similar challenges. And I just want to give a shout out to make sure that we don't skip it is that the data rescue toolkit project out of Johns Hopkins, they're also grant funded by IMLS and they're working on a toolkit to address things on the volunteer side, which, you know, is not the point of your question but I just wanted to give a shout out to them because they're our friends. And I just want to say that with the limitations, I think that's something that we started to understand about the collective impact model too is that we do have these distributed volunteer approaches. But a lot of what the collective impact model is about is measuring success. And so that's where we kept tripping up is because we don't know the full scope of what's out there, it's difficult to even begin to address what impact you're having and how you can continue those efforts. So I don't know that that answers any of the questions but just adds to it. And then to address the question about grants, we received an IMLS grant. We are looking at possibilities of other grant funding. We have not started to write nor pursue anything as of yet. And that's sort of we're trying to figure out what those next steps are so that we can identify the funding that would be most appropriate for carrying the work forward. And again, sort of to Ann's point, that idea of moving forward. We had, it was enlightening for us to have, at least for me, to have the data, the hacktivist, the civic data people involved in our forum because they bring sort of a different point of view of sort of let's get in and get it and this is what we can do with it. And there's a lot of expertise there that could really help us if we can layer the archivist or the librarian mindset on top of that as well. I think you could get a lot of bang for the work that they do. Great. Well, thank you so much. And thank you very much, all three of you, for taking your time to talk to us about your project. We'll go ahead and thank you for having us. Thank you.