 Hi, everyone. It's a minute after, so it looks like we've got a majority of folks that are joining us for the webinar. I just want to welcome you. This is the webinar on supporting open science, data curation, preservation, and access by libraries. This is a webinar jointly presented by two organizations, Center for Open Science and Internet Archive. We're very happy you're here. We're excited about talking with you today about a collaborative project that we've been working on to develop preservation workflows. And we're really interested in trying to drive forward with some feedback in this webinar and get your input on a few items. We'll be conducting the webinar using Zoom. And some of the features that I want to point out to you are the Q&A. So I know there's two chat features. There's the chat, and then there's Q&A. We would like to ask you to use the Q&A box to ask us questions or make comments. That way, we can leverage these throughout the webinar to answer questions. And at the end, we'll read through a lot of these to have a little bit more discussion about some of the questions and feedback that are coming up. So with that, I want to kick us off with our first poll, which is another feature that we're going to be using throughout. This is sort of a live polling feature that'll help us gather feedback on specific questions that we're really interested in learning from you as we work through some of the content we have to share. So this first poll is about what are the key priorities for preservation at your institution? And we've got a few ideas there that might resonate with you, some themes that we thought made sense. But obviously, if there's something there, that you don't see, feel free to pop it in the Q&A. And that's really useful feedback for us as well. So with that, I'm going to introduce the panelists for this webinar today. I'm Nikki Pfeiffer, Director of Product at the Center for Open Science. My colleague is Eric Olson. And I'll let him introduce himself. Thanks, Nikki. I'm Eric Olson. I'm on Nikki's team here at the Center for Open Science, managing one of the other products on the OSF for universities, labs, and other organizations called OSF institutions. And I'll pass it over to Jefferson from Internet Archive. Thanks, Eric. Thanks, everyone, for joining. I'm Jefferson Bailey. I'm Director of Web Archiving and Data Services at Internet Archive. And I'll hand it over to Laurie. Great. Thanks, Jefferson. I'm Laurie Donovan, and I'm Senior Program Manager for Web Archiving and Data Services here at the Internet Archive. And thanks all for joining. Great. So I'm going to kick it off. If you have recently joined, I think the poll is maybe still open or here are the results. If it's not open, there are just a reminder for new participants. There will be some polls over the course of the presentation. Please respond or if the answers, the autofilled answers are not enough, please feel free to put stuff in the Q&A for other answers to specific questions. We are talking about a collaborative project between Center for Open Science and Internet Archive. And this is a moment in that project where we're really wanting as much input and feedback from the community as we can get. So please participate in the polls and we'll have plenty of time for Q&A, either in the Q&A function in Zoom and we'll have plenty of chance to ask questions at the end. So I'm kicking off the slide deck. If you aren't familiar with the Internet Archive, we are a nonprofit digital library founded in 1996. The picture in the slide is our actual building. We bought a church. Our headquarters is in San Francisco. This slide is from an era when we encouraged people to come and visit. Obviously not a reality right now, but at some point in the future, if you are around, we do welcome visitors. We run a lot of our own data centers and the church is part of it is still a church and part of it is where people work. So it's an interesting place to visit and we encourage visitors. Our mission statement is universal access to all knowledge. People often are interested in hearing about how much stuff we currently have. At the Internet Archive, we did, of course, start archiving the web, but have certainly broadened our scope over the years to many other types of collections and formats. So there's the sort of number slide of all the stuff we have. Millions of these, millions of those. The web archive does constitute about half the collection. So we are approaching one trillion web documents archived since 1996 and the total collection is somewhere in the 80 petabyte range as a single copy. We, of course, have many more copies than that. So that's how much of the content. Our role within the Internet Archive and the web archiving and data services, we obviously do a lot of web archiving and data services. That includes both some products and services that many may be familiar with as well as customized web development or indexing or analytics services that are done on a more contractual basis. Archive, it is our sort of software as a service product for building collections of websites. It has over 750 institutional users. It's about 60 to 65% universities. Usually the university library or the archive or research effort or records department or things like that that are collecting for many different purposes, stuff that's on the web. And we do a lot with other library archives and museums and a mission aligned way as well as governments, not-for-profits, NGOs, all those sorts of things. We do many of these services or paid services. That's a big part of our role within the archive is helping support internet archive overall through paid services. But of course, we're a non-profit and archive.org and internet archive does a lot for free and you can upload stuff and you can of course use most of the collections, well all the collections I guess without paying. So we do have a sort of multi-faceted sort of cost model. Some things are paid, some things are at a cost, some things are free. For our group, we working with these seven to 800 institutions every year bring in somewhere around 1.5 to two petabytes ourselves as part of the larger organization per year and about five billion digital objects mostly from the web but sometimes we're getting them directly from non-web repositories, our content transfers or things like that. We do a lot of open source software development. We work a lot in the open data, open science, open research space as well and have many partnerships with mission-aligned organizations both through grant-funded efforts as well as through just strategic partnerships that might be co-development of tools and software or they may just be community building and advocacy efforts. So that's our group. And I'll hand it over to you. Thanks Jefferson. So you'll see a bit of a theme here or mission aligns well with our friends at Internet Archive and really at the high level our mission is increase openness, integrity and reproducibility of research and seeing that there's really not one without all of those concepts and integrating those into culture change in research. And we don't see that as just a technology problem or just a cultural change, catalyzation. There's a little bit of all of that involved. So our organization is actually structured to take that into account. So we have three distinct teams within the Center for Open Science. We have a policy team that explores the incentives to embrace change that organizations have already begun using or are considering and sometimes even consulting with us as they consider those incentives for researchers, a research team that is evaluating how all of these policies or other encouragements or technology are they facilitating or cultivating the change that we expect? And if so, how effective are those? And then we have an infrastructure team that Nikki and I are on that helps to build and maintain technology that enables that change to really take hold and be embraced by communities. And this is an image we like to use to frame that conversation. And one you've no doubt seen in other contexts before the diffusion of innovations and it really certainly applies in our space as well. And with open science adoption, not just the USF but open science practices and resources, we'll see that there are innovators they'll run at cool new tools or services that really seem to facilitate the practices that they are interested in or want to embrace early adopters that are following right behind them and trying new things. Most researchers or those in open or in science and other research practices are lumped in the middle here and they want to or are willing to embrace change. They just maybe need to see members of their community or other communities embrace those and try those first. And then of course some that need to see a lot of that need to have a lot of encouragement in order to move forward. And if we sort of tilt this and look at it as structuring culture change and enabling culture change that we were just talking about a moment ago, we see each of those teams that we were structured with in addition to how we operate within the scholarly landscape. So at the base of our COS pyramid here, we have the infrastructure, the technology, the USF in our case that enables change makes it possible to be an innovator and try new things. You have to have some tools to work with in order to do that so that the infrastructure makes our base. And then as those innovators come and try things and provide feedback like you are today, we can continue to innovate and iterate on that technology to make it easier to use which brings in more users that are excited about the possibilities but maybe needed a little more. The usability had to be a little closer to where they were in order for them to jump on board. And then once you start seeing that that influx of use and enthusiasm, communities can make it normative in that there is an assumption that this is real, this is now going to be a typical use case for us, data sharing or using data management plans or fair data, those sorts of things are embraced by communities and made or embraced across the entire group. And then sometimes it takes a little more than that to get all of the members of a community or multiple communities involved. So we see incentives like making data, shareable whole research objects, not just something that's cited in a paper as an example of an incentive that can encourage researchers to embrace new practices. And then finally, we see funders and institutions making changes to what they would even accept in terms of what is researched that is published or available as part of their research incentives like tenure. So we see policy change and making it required. That will get just about any part of the community that still wants to get their funding on board. And I think we're back to Jefferson to talk a little bit about this specific project that we're talking about today. Yep, thanks Eric. So this is just sort of an overview of the project. We are in the midst, I guess wrapping up maybe year one, so kind of in the middle of a two-year project to figure out ways to integrate Center for Open Science but really OSF the platform and Internet Archive for preservation. We were awarded an IMLS NLG National Leadership Grant in 2019 to support this work. I have a sort of the generic sort of mission-y abstract language from the grant application in quotations. But I think this speaks pretty well to what the overarching mission is which is leverage the intersection between open science research data, long-term stewardship activities and collaborative archives and then figure out how we can link these two up with downstream collaborative distributed data sharing and preservation services beyond just COS and IA. So the actual work in the grant is two-fold. There's technical work to actually automate the preservation mirroring from OSF into Internet Archive and then of course training, testing, use, community input, community feedback, reporting out about how this is happening so that it can very much be community-driven on how we make many of the technical decisions. And so this webinar is part of that. We will then, after we have sort of finished some of the technical integration work between our two systems, we'll, we have some other partners that will do sort of prototype very low-scale integrations to then get the data from IA into other preservation systems that people might be using locally. And then lastly, we'll of course remap further integrations beyond just the content and scope for this project. So I just figured I would talk a little bit about some of the preservation partners specific to this grant and the ones that at least we, Internet Archive also work with. So we do much work with LOX. That's lots of copies keep stuff safe. The preservation network that many people are a member, members of you can establish sort of, it's a network-based replication protocol fixity checking thing. You can set up your own network. They also have a global LOX network. We, Internet Archive and some of the services we provide we do replicate the data that we are archiving to LOX on behalf of partners that want it. We have also done many technical development collaborative technology API development kind of projects with LOX too. So we work with them quite closely there involved in this project. So we will be putting some of this data that is totally open or CC0 or whatnot into the LOX network just to make sure to test the redistribution of the preservation content into LOX. We are doing the same with AP Trust, Academic Preservation Trust. This is another collaborative digital preservation network like I think around 35 or 40 academic institutions who we also have worked with on other projects. And so they will be taking some of the OSF content and his merit that is relevant to their member institutions. Another project partner is Los Alamos National Library. They actually run their own instance of OSF within Lannell. And so they provide a very interesting use case that we wouldn't really be able to do otherwise of how to test our integrations on a localized OSF instance. So they're also a sort of additional partner. And then also the PresQT project at Notre Dame will also be taking some of the OSF data and testing it in their workflows. I put some others on here that we've worked with both mostly for open data or open science projects. DAT is a decentralized web protocol. We had a pilot project working with DAT and some of the data in California Digital Libraries open data system. So that was very testy and prototypy and we have other decentralized web partnerships just to test and see how it works to get totally open science or research data into some of these D-web type systems. And we also over the years have worked very closely with DuraCloud, which is part of DuraSpace, now under the Neurasis umbrella. And many of them sort of have a preservation management system that is on top of some of the commercial cloud services that many of our joint partners use. And so we have some automated systems for getting data that IA may have collected or taken a copy of into DuraCloud. So just to dig into one of the examples of those, especially LOX, which I think is maybe the most relevant to the most number of people maybe on the call because they have a pretty broad partnership network as do we. Many of the internet archive partners use us for either collecting or for having a copy outside of their own institution, mirrored at internet archive, and then are interested in having it mirrored maybe if they end up becoming a member of a LOX network. But for most of them, they're collecting using our tools in internet archive. And then while we of course keep copies for them, they also want these copies in other places. And it's often in the LOX network, so we've done many integrations. This is just a little diagrammatic that is what that workflow looks like. They're usually harvesting things either from the web or maybe they're using our digitization services at IA. These digital objects or files or whatever they might be, then of course go into IA storage and I'll have a copy that talks a little bit more about our storage and preservation work after this. We then can do two different things. We can either distribute that content automatically into LOX networks that are outside of internet archive. But we have also in the last couple of years work with LOX to host our own LOX nodes within IA. So we have LOX nodes up and running that we can link to an existing private LOX network or that can be part of a new private LOX network that is, and we're just a node in some sort of private network that others are spinning up. The advantages to that are that obviously we can put the data there pretty easily because we're already running the LOX node. It's separate within our infrastructure but still within our infrastructure it's much lower cost. And then it is then automatically networked using the LOX protocols and just how LOX works. It's automatically networked across a whole LOX, automatically replicated across a whole LOX network which means that the institution that we don't have to put it into a network ourselves or an institution takes the copy and puts it in there. So there's a good deal of saving of effort of mirroring and shuffling data around by that. Then as people continue to use our services continues to happen and then new nodes can come online. So what else do we do at internet archive around digital preservation? Some of the advantages that we have is we do of course run own run and operate all our own data centers. We can put things in the commercial cloud if someone really wants us to. But for all intents and purposes we are a cloud provider. We just happen to be a nonprofit one. We don't make money on it. We do it at cost, all those things. And we are really only working with mission aligned organizations that are nonprofits that are advancing open science that are advancing library and archives missions or social good or social impact. And we can do a lot of these things end to end which is not just collecting, indexing, building people websites to her discovery of their stuff and then replicating data into a preservation networks. We are far cheaper than the commercial cloud. We don't charge for bandwidth. We do have hardware costs and it does cost some money to store stuff. But we also get some exemptions around some of that. We have a lot of engineering capacity, both people but also just the physical infrastructure. And we do have, as I mentioned, hundreds and hundreds of community partners. So it's very easy to do partnerships and integrations and things like that. We're very API first. We try to operate in systems interoperability mindset instead of trying to building things from scratch. And yes, we do have a sort of global reach and lots of partnerships and collaborations. Next slide. What do people ask us to do beyond what we do out of the box? We can do as many redundancy copies as people want. So I don't know what the most would be but we can do one million copies of a super precious special file. But certainly more than two is something that we do. Customize, fixity check frequencies. We put things in multiple storage architectures. So it's agnostic to specific storage technologies. We have data centers outside of the United States as well as ones outside of just the Bay Area, our main location. So we can do geopolitical redundancy and distribution. We mentioned some of the automation aspects. So we don't charge for ingest or egress which is how the commercial cloud makes most of its money. So the accessing files really doesn't cost anything. It's mostly the storage, the permanent storage and just the infrastructure cost. And as we mentioned, we're a nonprofit. There's just a little dashboard that we made because it helps to visualize what some of these things look like and some of the services that we provide. So a lot of the internet archive is done like a very minimum viable product which is totally sufficient for many of our partners who might be tiny, tiny organizations and ones that pay us for more advanced work or for higher fixity checks replicas. We have specialized services really and UIs for those. So that's it for me. Oh, yep, so our project timeline before I handed on current status I mentioned we're about halfway through. We have developed the prototype. So that is the automated transfer of registration data and Eric will talk a little bit more about that from OSF to IA. So we have done the technical work to automate the transfer process and checking that and how it happens and the frequency and issues that may come up just in the transfer of data. And we are basically at this point of getting feedback on the prototype. So engaging both the researcher community that's creating a lot of the data as well as the preservation and the library archives community that wants to steward this data further, take copies of it or at least be involved in the decision-making process around metadata issues, metadata mapping, format conversions and just how the long-term preservation actions will happen, whether they're done by internet archive or whether a copy is taken and they're done locally. So that's where we are. We're gonna after the feedback gathering stage we'll update the prototype. We'll production launch it. We'll do all the dissemination around training and documentation and things like that. And then sort of the last stage of the project is distributing it to additional preservation networks, the prototype work I talked about as well as exploring additional content sets or building integrations around. So more to come. All right, thanks, Jefferson. So as Jefferson just mentioned, the key component that is gonna be moving from OSF to through this internet archive project are registrations on the OSF. And if you're working with or you're part of a few different research disciplines, registration is something you've used or heard of in the clinical fields or psychology, whether that was on the OSF or not. And the registries are a particular object within the OSF. And I'll show you what one looks like in a moment. For those that are not familiar with registrations they don't really replace different kinds of research outputs. Instead they supplement those and a registration creates a permanent time stamped version in the OSF's case, your project, your files so that you can describe what your project intends to do, what your hypotheses are, what your data gathering and your other methods are going to be before you go and do that work so that you can demonstrate transparency through that research process. And the value of a registration really will depend on how much information you are including at that time of registration. So that minimum we're hoping that you include a research question, your population and your sample size, your general experiment or study design and then variables to be collected in data sets that you'll be using. And there's some variability to this based on exactly what you're registering and how you're doing so. But generally that's what registration is expected to have. And there's a few reasons to do that. I mentioned sort of generally a practice of transparency in your design and publications of your work. It also helps to reduce the file drawer effect so that you can register a study that maybe the data doesn't pan out and it didn't have a real sexy result. But that's okay and that data can still be extremely valuable. It may not get published in your first choice but it doesn't have to go away entirely. It could be as a registration needs to be a really valuable data object that others or yourself may continue to use and you can cite in that registration. And then the registered analysis plans, again, depending on how much information you include in your registration can help improve the study accuracy and replicability by guarding against that the false positive inflation to help improve your chances of publication. So that's not really helping science move forward. Obviously it doesn't help with replicability. So registrations can help to prevent that. So I've set up an OSF project for us today and this link will be in your slides. You can go and visit it now as well. And we're gonna look at this project and set up a registration right now live so that you can see what this looks like and then Lori will sort of take it from there and show us what the rest of this project life cycle, this collaboration project cycle looks like. So this is our project for today. You can see all of us here is contributors and our DOI. And one of the modules here on our OSF projects is registrations. And I've already actually submitted, completed one registration for this project. I used a template called OSF preregistration. This one happens to be an embargo because it's not ready to quite go public in that case. We're gonna start a new registration so you can see it beginning to end. And one of the first things you'll notice here there's several options for a template for submitting or for completing my registration. And these vary depending on their purpose as well as sometimes the discipline or publication or destination of this work. So you'll see some specific locations or affiliations with a few of these and more like that to come as some communities start defining their own templates for registrations that will be available on the OSF soon. Today we're gonna use an open-ended registration just because this one is an opportunity to just do a narrative description of what our study will involve just so we can go quickly. We're gonna keep our title today from our project and we have an opportunity to define what kind of object this is. So if we're only registering our procedures today we could do that in our case we're just gonna keep the project. I have an opportunity to set an affiliation of this registration because I personally have multiple registration or multiple affiliations available. I'm gonna go ahead and keep my Center for Open Science affiliation present and choose a discipline and a tag. Use the preservation today and proceed. And with several of the other formats we'll see significantly more specific information requested because we're using this narrative version it's much shorter in terms of how many sections are requested but still the idea would be to include the necessary information on today we're just gonna add a paragraph about what our goal was for the webinar today and proceed. We'll have an opportunity to review our registration and then upon completion of an opportunity to embargo for up to four years like I did with our other registration on the project. In our case, we're just gonna go ahead and make this registration public. And it takes a moment to archive based on how much information you've included but this will now be a time-stamped version of our project with the new narrative that I've included for our registration on this specific project. So with that in mind, and I know that's a whirlwind tour of a registration but we do want to get some of your feedback again on a new poll that we've just shared to get a sense of when you would expect the registration to start the process of the life cycle of this collaborative project with Internet Archive and be archived. Would you expect it to be initiated by the researcher separate from the registration walk-through that we just did? So I could go to that registration that was already completed and claim at that point that I would like it to be archived or would you expect or prefer that automatically when I complete that process of registering the project as we just walked through? At that point it is automatically archived. Obviously there's gray area there perhaps and you can tell us a little bit about if you have a different expectation in the Q&A box but if one of those two extremes would align with your expectations would give us an idea of what you would prefer or expect. We'll leave that poll up for a moment while I pass it over to Lori. So I'm gonna put you over to host a few. Thanks, Eric. Are you able to claim it, Lori? I think it's once you stop. I can start. There we go. And let's see. So not, let's see. Is the poll still going? So what I'm gonna go ahead and do is show the part of the process basically how the preservation happens or how that gets into the internet archive. And if you, you'll get a copy of these slides at the end so you'll be able to sort of click on these links that we've shared and be able to sort of access some of these sort of beta examples of a registration preserved on archive.org. But I'm going to walk you through this in the browser. So this is sort of an existing registration that we used as an example of the process. And Eric very nicely walks through the process of kind of adding this overview information. And there can also be sort of files and other components that are a part of the registration. And what we do is we take all of that information and bundle it together using bag it and preserve it on the internet archive. So the overview information we grab via API and make a JSON file of it and all of that information populates into an item on the internet archive. And this is what it looks like. The goal here is preservation, not necessarily access or sort of the front end piece of it. But you can see overview information about this registration. You can see a link back to the nice kind of pretty OSF page for this registration. And you can see here there are a few associated items or registrations that each have their own page that looks just like this and each has their own link back to the OSF registration page. You can also kind of view the metadata behind this. So information about what is part of this item. So information about each of the files as well as sort of the high level metadata description that's shown on the archive.org page here. If you want to see all of the files, if you wanna download them for example for local storage or for integration with other external systems, you can click to see all of those files and they're in sort of a directory structure. So you can see kind of a full list of what's included and you can kind of click in specifically to the data for example and see what this JSON of the registration itself looks like. So this includes all of that really kind of important information that was added during the registration process that Eric showed. And you can kind of see high level here. This is the sort of sandbox collection for what we've pulled over so far in this sort of beta testing period. And you can kind of imagine that there could potentially be collections of registrations that are affiliated with a specific organization or other ways to sort of group registrations together on archive.org for preservation purposes if that was of interest. So those are kind of the basics of how this content shows up in the internet archive and sort of the preservation aspect. We have a couple more things to go through including another poll before we wrap things up and have some questions. So here our goal is to get more information about what registration metadata is most important to you and feel free to choose as many options as apply. There's a lot of different types of metadata that's collected and they may have different levels of importance depending on your goals and your sort of institutional use cases. We'll give it a few more seconds for folks to choose the applicable options for them. And we have gotten some questions via Q&A and we'll mostly I think answer those towards the end. It looks like descriptive metadata is the most of interest but there's a pretty wide disbursement of options there preferences, let's see, great. So a little bit more about some of the upcoming and future work that we have going on for this project. We're going to continue gathering and integrating feedback, having you all here, providing feedback, answering polls and hopefully having some lively discussion here shortly is a really important part of this process. We'll continue to refine the prototype to incorporate feedback that we receive and we'll do some further systems integration testing that Jefferson talked about with some of our partners and do a production release. There's also UI work, training materials. We wanna make sure that people know that this option is available, know how it works, get some additional sort of outreach out there. We will be pushing data from the internet archive to external preservation partners and systems that Jefferson mentioned and then we'll be doing some specification gathering, some planning for additional OSF content. Integrations, so beyond registrations alone. Okay, one more poll. So what would you do with the registrations once they're on in the internet archive? And there are a number of different options, things that might be of interest. And again, with any of these polls, if you have an idea that isn't represented in the options, do chat that in because there may be some that we hadn't considered. There's always a variety of use cases in any group, right? Let's see, it looks like most, the most common option was ingesting into an IR or an institutional IR and a lot of breadth across the options there too. So thanks for answering the polls. And then we have some other sort of open questions or things that we're still kind of working through and where we could use some additional feedback about sort of how to curate this content more once it's on the internet archive site, what folks' goals are around that. What are some of the metadata and protocol needs? We asked about in the previous poll about what metadata is most important, but more specific information about needs around that is really useful. Sort of different metadata and other types of protocols that we might use or want to support how the data goes into IA, whether it's a push or a pull model, how to deal with updates to the content, how to deal with other types of scholarly content, dealing with things like pre-print, status sets, et cetera, and how to make this really extensible. So these are some of the things that we're interested in getting feedback about now and in the near future. So I think as we're sort of getting through our time, we'd like to really open it up for questions and feedback. We have all of our information here, and I think one of the first Q and A questions that we got was about who to reach out to at IA around locks integrations, and Jefferson or I, the first two options there, are good folks to reach out to about that. You'll get follow-up, email, poll results, webinar slides, additional resources shortly. So in case you're wondering about that, but we'd like to open it up, and I think we'll start answering some of the questions that came in over Q and A. Yeah, I think one of the questions mentioned in here, I think has been answered, but I wanted to elaborate or spend some more time on it. And that's the question about, is the intention to have files or the contents of these files associated with the registration archived as well, or just the HTML content. And I think from what Lori demoed, you were able to access files, but I don't know if you wanna pull that back up to show that off a little bit more in detail and see if Matt has any more questions about that, because the goal is to archive all of it. So to preserve the registration template, so the questions and responses that are part of sort of that pre-commitment of a preregistration or any registration type, along with any files that get uploaded or exist on that project at that time to create that timestamped, immutable version. So I hope that answers the question, but I'm happy to see if others wanna chime in on the Q and A with any further parts of that that we can actually dig into and show off right now. The example that I used didn't actually have any files associated with it, but they would be there if there were files associated. Yeah, I think it has like a TXT file or something. Yeah, so yeah, not the greatest example for that. Yeah, every file and there would also be a metadata file or a metadata JSON metadata file about the file. So any of the data would show up there. And I think we mentioned before too that we were using the Bagot protocol for how we are taking an OSF registration and sort of comprising it into a package that can be handed off. So I don't, I think we said that, but if that's useful to understand how that's actually happening on the backend, that's the format we're using. Yeah, the question may have been sort of like, are we crawling it like a web crawling web archiving kind of sense and the answer is no, no we could, but the integration is to APIs. Which also will mean that if a registration gets withdrawn, we will send an update over to Internet Archive about the parts that are no longer relevant and what should be removed. It looks like there's a question in here about how does the Internet Archive work with Europe's right to be forgotten laws? I will have to defer to Lori or Jefferson on that one. Yeah, we do, you know, we are a US based institution, but we do have European partners. So this does come up, right? Right to be forgotten is a little weird and then it's not exactly like a legal mandate, but the general high level Internet Archive policy as with many other online platforms probably Center for Open Science as well is that we do have teams that are responsive to take down requests. So, you know, if people are, you know, right now this project is working with open content, right? So it's, you know, the users are sort of determining in advance that it's open and doesn't meet that, but if it's some later point it did, then we have mechanisms and teams and workflows that can respond to take down withdrawal, but it's not necessarily a legal obligation or what's those times fixed? I think we have a clarification to see if we can answer with withdrawal of content from IA for a withdrawn OSF registration, be automatic or manual and that would be automatic if it's one that's then pushed then it will be updated automatically. Yeah, so that's part of the integration is that actions that happen in OSF will impact the part of the IA version that may already have been replicated. I think the one question that sometimes comes up around that as well, like overwriting data or something, if data is edited, post-preservation ingest, generally since the data in scope for edit, which is usually just a metadata field or something is pretty trivial, we would actually just make a new copy instead of overwriting it. So our approach is archival, we don't generally delete much of anything. If not, if it isn't necessary to delete it. So for when it's like overwriting metadata, we would probably just version control or versionize the relevant JSON metadata blob. So that the current one would be current but the former ones would still be there, if that makes sense. And we also have a sort of a temporal question from our friend Natalie Myers in the chat. At the point of archiving, is it publicly available on both endpoints, on OSF and the Internet archive endpoint? Yes, I believe so. If it's up and on one, it will be on the other. Yeah, it's a mirroring and not a transfer, just I think that's the point of clarification. So it'll be available at both places. Well, we're certainly interested in hearing from folks about the metadata stuff. I think for registrations, just given the, not complexity, but the nature of the hierarchical componentization of it, there is a lot of metadata. And you can even see like in Lori's demo that there's metadata basically for every object and often for the transaction itself too. So it'd be interesting to hear how people work with that locally, you know? There's a lot of that you could probably get from OSF itself and even more, you can probably get from the IA mirror copy. A lot of that would not be useful if people are just interested in preserving the sort of canonical parts of OSF registration or project or whatever it might be, but some of it might be. So I think one of the things we're able to get through this webinar and definitely through this phase of the project is input on that aspect. Yeah, and we're winding down on time, but there's a couple questions related to user account security and specifically around hacking someone's account and then removing some of their research content from the OSF. I'm happy to follow up with that person more to understand the complexities that you might be dealing with. We do have security measures in place on the OSF accounts with two-factor authentication to ensure that no one can log in as you if you have that enabled. Additionally, just a couple little tidbits that we rushed through or sort of doing the quick demo of a registration, but to create a registration, if Eric and I were doing a collaborative research project, we were both administrators of that research. If I were to initiate a registration, I could push it through to archiving, but then it needs his approval and he could say, no, I'm not ready for this to be made public and all of my research protocols being out there publicly and he could actually not approve that registration to be made public, which would mean it would not move forward. If he didn't respond within 48 hours, then it would move forward. You can also embargo registrations for up to four years, which would mean that wouldn't be made public immediately. The same process actually takes place when a withdrawal is requested. So if Eric and I did approve that registration, it's made public, but Eric decides he wants to withdraw it for whatever reason, I would also get that email request again to approve withdrawal of that registration and there's my opportunity to say, wait a minute, I didn't agree with that and I don't want my research taken down and therefore I could not click, I could actually cancel the withdrawal or if I don't then within 48 hours, it's like I silence his acceptance in some way and let it move forward. So those are some measures that we've put in place to help protect authors from either exposure of research that they're not ready for it to be made public out there or same thing on the other end where something could be taken down that you didn't want to be removed from public view. So hopefully that answers those questions. I'm again, really happy to follow up with that person down the road to make sure that if there's other specifics that need to be thought through that we're happy to consider those as well. It seems like all of the questions have looked at, have been answered and so I really wanna thank everyone for participating and listening to us describe the work that we're doing and hopefully providing some good awareness around this project and potentially some value down the road to each of you and your institutional needs. And to also help us with the feedback you've provided those polls will be something that we actually spend quite a bit of time on evaluating and doing some analysis on and incorporating into the updates that we're gonna be making. So we're very appreciative of that and for your time, as the slide indicates you can reach out to any of us with any of these questions, especially during this live Q&A. It's really difficult cause you can't talk to us that we're sort of inferring what we think you mean by some of the questions and so certainly reach out and we're happy to continue the conversation after the webinar ends. And there will be a follow up email with more information. So with that, I just wanna thank everyone and say goodbye. Yes.