 Okay. I am sorry for the technical difficulties. My internet either just went down or my computer just went down. Either one. Someone maybe need to jump in if that happens again. I apologize for that. My name is John Shadacki. As I was saying, I'm at California Digital Library. I'm joined here with colleagues from California Digital Library as well as ARL, the Association of Research Libraries. And today we're going to be discussing a pilot program that we have from an IMLS funded grant regarding machine actual data management plans, or as we're beginning to call them data management and sharing plans. And so I'll just kick us off at the beginning with a little bit of introduction and housekeeping. And we will move into presentations from other members of the grant team, the project team. First off, we just want to remind everybody that there is a code of conduct and we will be following the ARL code of conduct. Judy just put the link into chat. Please take some time to open that up and review and make sure you understand the rules of engagement that we have here today. As I mentioned earlier, we will be using the Q&A feature within Zoom. If you have any questions throughout the presentations, please add them to the Q&A feature here in Zoom and we'll click those and we'll address them using that feature. So you should be able to find a Q&A button at the bottom of your screen. Also, please make sure that you keep your computers muted. We do have multiple presentations and we just want to make sure that we don't have a disruption of background noise from other speakers. And the last thing, as you saw, we are recording this presentation and we will make the recording available online after the event. So the agenda for today, this of course is the welcome. So thank you for joining us. I will do a quick project overview and then that will be followed by a discussion by Maria on what is the current state of practice around machine actionable DMPs? What is the landscape around the tools that exist and the approaches that exist, including DMP tool? And then we will jump into a presentation by Cynthia about the specifics of the pilot, the pilot project that we have from our IMLS grant. And that will be followed by Q&A. And of course, if you have any questions, as I was mentioning, please put those in the Q&A feature here within Zoom. So just to kick us off a little bit about the project overview, as we mentioned, we are able to do this from the generous funding from IMLS. And so we really appreciate their support. This is an IMLS funded project, but it really builds off of years of work that's happened throughout the community when it comes to data management plan and the transition to more machine actionable DMPs. And this, of course, is built on the kind of evolution and revolution that's happened across libraries, where we as libraries have had to jump into RDM and data policies and really support researchers in the wider community and this transition to a more data-centric and data-publishing-centric view of what research outputs are. And we know within the library community, we spend many of our hours on consultations and discussions with researchers about data management plans, the policies that surround them, the best practices of surround them. And we are familiar, many of us, with the tooling that is there to make that possible, the guidance, the kind of drafting of DMPs, but also the follow through and the best practice guidance of how research should be done. It's something that we are well versed in and have had to become well versed in over the years. This pilot project that we have here is really about thinking about how to take that knowledge base that we have in the library community and the connections that we have built throughout the institutional community and on our campuses and at our institutes and try to think about ways to leverage a more machine actionable version of DMPs within our own local contexts. So what we want to do here is with the funding from IMLS is to take the infrastructure that has been built from DMP tool and other DMP providers, tool providers, around approaches to machine actionable DMPs. And Maria will get into this and pilot how those can work in local contexts. And so the goal here with this project is for us to is to find institutions who are willing to jump in and demonstrate value, not just to their communities, but also use it as a demonstration case for all of us to be able to look at and leverage and use as advocacy for this change that's happening across our communities. And so we'll kind of move right now into a description of machine actionable DMPs from Maria. And then we'll get into a more in-depth discussion about the pilot itself with Cynthia later on. So with that, I'll give it up to Maria Fritz Ellis. Thanks, John. So yeah, I'm going to go into some more details of exactly what machine actionable DMP is, what things you can do with that information. I'll give a little bit of information on the background to how we developed it. And finally, some details about our API. So starting at the beginning, exactly what do we mean when we say machine actionable data management and sharing plan? We're talking about transforming that traditional static, usually two page, usually a PDF narrative document into a structured machine readable version of all that information. So we can take it and share it machine to machine. And really our goal with that is to enable automation. We want to facilitate integration with administrative and research workflows, which is what we're talking about today. Another important thing to know about machine actionable plans is that they really need to be living version documents that are updatable over time. So this is another aspect of these of this workflow that we are looking to pilot out with institutions such as yourself. So I think it's important to think about why are we talking about machine actionable plans right now? And although CDL, we've been working in this space for many years now, I can say within really just the last year, there's been an increasing interest in using machine actionable DMPs and more conversations about the possibilities with it. And that's really coming out of increasing mandates from funders for not only the creation of machine actionable or data management and sharing plans, but for the really for the sharing of data as well. So as there's this new emphasis, institutions are thinking about first of all, how do they help researchers who are faced with these new requirements? And how do they monitor and track to make sure that their institution is in compliance with these new policies? So really building on that and responding to that community need, we've focused on building a new kind of robust research data infrastructure, we're working with all with open community systems, and we're trying to utilize existing ecosystems and existing technologies so that we can best and most efficiently integrate all the good information in these plans into other systems. So what can you do with a machine actionable plan? So at the highest level, what we're looking to do is to exchange information about research and communicate it across stakeholders. So this could be information from library from plans going to libraries, it could be to data repositories, IRB boards, publishers, IT security, all of those parties that are interested and need to know about research that's happening at their institution. So what we're looking to do is facilitate the easy retrieval of information about research in a centralized place. We're looking to facilitate notifications and verification for important pieces along the lifecycle. And we're looking to facilitate compliance checks. So looking at what what policies a researcher was working with when they received funding and what data was shared at the end of a project. And at the end of this really the goal that I try and keep front of my mind and I think is really forming the basis of a lot of what we're doing is we're working to lessen burden. So we're not looking to build out new complicated workflows or new additional work for researchers or administrators. We're trying to build as a automated and robust system as we can with that with the goal of really lessening burden and increasing access to good information about research. So the this the machine actual plan and the work to kind of facilitate this transformation is very much an international effort. This is not something that is only within the California Digital Library. There are many other providers similar to the DMP tool internationally that are also working with us in this space. So we have worked together kind of as a as a tool provider community to coalesce around standards for how we express plans in a in a structured manner. And that's really important because that means that we can all agree on exactly you know how we should be structuring and expressing this information. So we are truly interoperable with other systems. So thinking about you know what potential integrations or projects that you might be running to are interested in working on. I think you need to consider what metadata is in a plan. So all funders are somewhat different as far as the questions they ask in a DMP. But generally speaking they all cover these five main components. Data type very could be very important could include formats include size and volume security issues as well tools and software. This is increasingly important as more and more analysis is reliant on access to specific software standards looking at metadata standards instrumentation that's used in the process preservation and access. This is the section that I think has the most kind of meaty information that is could be most useful for institutions to share. So this section usually contains information about where a researcher is intending to deposit. It might include information on any licensing issues. If there is bargos if there is security issues around data could be ethical issues maybe surrounding how they're going to handle working with indigenous data. PII all that information would be contained within this section of a plan. And finally most of these plans ask about oversight. So who are the individuals that are going to be involved in data curation as part of your project. So this should include the names and roles of any of all the individuals who are working in that that area of any given project. So other things that you can do kind of with a machine actionable plan. First of all as I was saying facilitating guidance. So helping librarians and administrators really work with their researchers at scale. So get a sense of all the research that's happening at an institution at the time so that they can provide guidance to the most in the most efficient manner with the fewest resources. That's kind of what we're trying to achieve there. Again compliance. So looking at what was stated in a plan and then what was eventually shared out as part of their research. Also promotes research integrity. This is really important in terms of creating open data practices and research reproducibility. Also data security. And finally tracking impact. So helping administrators and other officials really look at the impact of an organization's research programs by giving a sense of what research is being conducted currently right now and then what outputs have been generated down the road. So digging a little deeper into this pilot and thinking about what types of integrations you could do or ways in which you could use this information. These are the ones that come up the most. This is by no means everything. People might have other ideas for what they want to do. And we're completely open to that. These are just the ones that I hear the most often. But I'm certainly interested in hearing other ideas. So some of these things include notifications. So getting alerts if sensitive data is going to be part of a project. If maybe there is going to be a very large volume of data that needs to be deposited somewhere can help with enabling data transfers. Some people are also interested in connecting this information with existing systems and other systems like RIMS could be faculty profiles as well or grant internal grant systems so that we can exchange metadata either sending it to the RIMS or possibly RIMS could send it to the DMP tool either way could work. We're also looking to engage departments in testing using machine actionable plans in research workflows. How that can help with data management needs. And finally looking at improving communication workflows. So improving the exchange of information between say the grants office, research office and the library or possibly between the research office and IT and security. So making sure that all of those parties have kind of a platform, a connecting platforms to exchange information about what research is happening at any given time. So I thought when we're talking about machine actual DMPs, it can be a little bit abstract. So we do have a landing page. So an actual page that you can point to with basic information about a project. So I'm going to click in to this. And this is the landing page for a specific DMP that was created in the DMP tool. And up at the top, you can see the persistent identifier for the plan, you can see the version number for this plan. Up here on the right, you can this is where if a plan is public, you can go in and you can actually read that two page narrative document. It's really important to know because I always get asked plans are private by default. It is up to a researcher to make their plan public. We love it when they do. We encourage it. Most researchers don't. So just so people know. So if you had chosen to make this a private DMP, you would have all the metadata would appear on this landing page, but you would not have access to this narrative on the right. So digging into this, we've got all the contributors, their role specifically within this project, their affiliation. This is a persistent identifier behind that. It does link to orchids automatically as well. So high level details about the project. And this is information that we're really trying to grab from awards API. So especially for funded projects where you're trying to generate a machine actionable DMP, we can check the funder API in just as much information as we can in an automated way. This is where we have the planned outputs. So this is where we have a lot of really good information about, you know, what outputs that they intend on generating as part of this research. There are other data points that are available. These are just the ones we show on the landing page, but we actually are recording more detailed information in the application. But you can see it says, you know, format, we've got the size, the intended repository and license. We also record information on sensitive data and PII as well. And finally, at the bottom, the other works associated with this project, this is where once a project has been going for a few years, and it has generated any kind of outputs, we are able to we were working to be able to identify those outputs and then connect them back to the plan. So that's where they would appear on this landing page. Just going to head back. Alright, so a lot of folks have asked for information on our API, particularly people who are interested in doing technical integrations with us. So our new pilot project really features driven by a very robust API that we've been working on for a number of years. And you can use this to integrate your organization's data into your own local systems. And vice versa, if you want to ingest information into the tool, you could do it that way as well. So our API currently supports the retrieval of information, you can also create plans through the API, you can edit plans through the API, and we can share that data out as well. The other way we're using this API that's really important is how we are monitoring external systems and funder APIs. So this is where we're getting information on outputs, we're also getting as much award information so that we can pre populate whenever possible. And really the API functionality is one of our primary focuses, as we do go through this pilot project. So you know, really hearing from specific use cases about what information, you know, an institution needs, what format they need it in, so that we can make sure that we're sharing it in the most kind of useful way possible. And if you have detailed specific questions about the API, I'm happy to put you in touch with our developer, and we can really dig more deeply into, you know, what you're interested in working on. We also I have a QR code here, if you're interested in looking up our API documentation, as well. So currently, we are working on this is kind of where we are in terms of our overall work in machine actionable plans. The pilot project that we're talking about today is really important for us, as we really test out some of the workflows we've been working on, and we expand on them as well. We're also focused on connecting and tracking outputs. So we're going to be developing additional integrations with funder APIs, additional integrations with indexes and aggregators of the research network, such as open Alex. We're also in the backend, developing a completely new data model for the DMP tool. And this is really important, because this will allow us from the beginning to create as good rich structured data as possible, and facilitate communicating that information through machine actionable plans. And finally, we're looking at experimenting with the use of AI and machine learning. So with machine learning, we're looking at doing entity extraction. So taking a DMP that was not created in the DMP tool, it's just a narrative PDF document extracting as much metadata as we can from it automatically, so that we can structure it and transform it from a static plan into a truly machine actionable, structured plan. We're also thinking we're also working with AI to facilitate the generation of plans. So thinking about how could we use AI to help researchers say find the best repository for their data, or information about the specific policies of the funder that they are creating their plan for. So thinking about ways in which we can use AI to help researchers create good plans as easy as possible. And so that's it for me, I'm going to hand it to Cynthia, and we can certainly we'll go through any questions you have at the end. Thank you. Thank you, Maria. And hello, everyone. It's lovely to see you. I'm just going to spend a little bit of time kind of highlighting the pilot, and then sharing some information that we weren't able to kind of all fit in the expression of interest. So overall, you know, the goals are are multifaceted. It's to test the MA, DMSP features and functionality and an institutional setting, as you've heard about. It's to pilot the integration or the creation of prototypes and possible workflows. And then one of the bullet points I forgot to put in here, but I think it's really important. It's also to help you at an institution kind of extend and test your own capabilities for research data services. And I'll touch on this a little bit more in a minute. But I think that when we are thinking about crafting a pilot project for this work, it obviously has to be kind of like mutually beneficial. And I think that's why you saw kind of like a range of pilot examples there. So the term for the pilot is from January 2024 to January of 2025. And so the idea is we'll kick things off in January. And I'll go into detail a little bit more about what that whole year will look like. Because I'm sure you have some questions about that too. One of the things we are interested in receiving proposals around is to be specific kind of like in the team makeup, you know, given the fact that research data management and sharing is, you know, across institutions, service and responsibility. We are seeking proposals from institutional teams that kind of can kind of span the various stakeholder offices, such as the library, and the IT and research office and others. Obviously, if that's perhaps like, not in your capacity, but you still feel like, oh, it'd be really great if we could do a pilot with this regard with just library or just IT or just research office, like, we're open to seeing those proposals as well. But we just felt it would be more successful if we could have kind of these these connections and communications early on. Next slide, please. So Maria touched on all these, I'm not going to go back through them. Again, I think the thing to notice here is that there's highly technical kind of integrations happening here, like sandboxing with, you know, research information, management systems or data repositories. And then there's more kind of like socio technical like wireframing and, you know, creating workflows and making connections and communication channels and all those things which are equally important if you're going to like move on in the next stage to a more technical integration. So I would just say as you're kind of developing your proposals for this to do something that benefits your institution. So if you're at the point within your research data management and sharing services where you're just trying to kind of like strengthen relationships with your VPR, your IIT, you know, a more a less technical approach, maybe, maybe better for your pilot, right? So if you can do something about or pilot something around communication workflows or better understanding how they would like to use a MADSMP, all of that would be really, really fascinating. Applications will remain open until November 10. Then we as the expression of interest indicated, we anticipate making notifications by November 21. Next slide please. So as I mentioned, it's a year long pilot program. We will kick things off in January of 2024. We have probably a half day or full day of a virtual kickoff meeting. This will bring all members the pilot cohort teams together to kind of learn more about MADSMPs, how they may be used, and then share information about like the institutional infrastructure and organization. From February until December, we're going to have alternating virtual cohort meetings where at least one representative from like the pilot cohort will attend a bimonthly entire cohort meeting, I'm not saying that well, where you'll have the opportunity to discuss, inform, and then share information about what's happening at your institution with regards to MADSMPs, and I'm going to provide a better example of that in a minute. And then in those off months, yeah, every two months, thank you, Matthew, bimonthly is every two months. And then in those other months, we'll have office hours for those institutions that just kind of have questions about for those pilot institutions that have questions about the MADSMP who maybe like want to talk with a developer who have some some just some things to kind of like work through with the project team. From August to October 2024, the project team is going to be doing site visits at each of the pilot cohort institutions. And the goal here is really to just support the socialization of MADSMPs at the institution and to have some conversations with folks who are working on them, or working with them about like what features and functionality they may want to see or or just better understand the institutional context really. And then finally from November to December, we will ask folks who are part of the pilot cohort to draft and publish a set of case studies or best practices from the pilot projects. The goal here is really to help others learn and to share the experience out. I think even if you know, I think everything within this because it's a pilot, there's there's stuff to be learned and to build from. So that's really the goal there. Next slide please. Yeah, thank you. Sorry, we are looking for four pilot institutions. So the bi-monthly cohort meetings, I just wanted to give you an example of kind of what the format is. So this is what we tentatively have planned for March. There'll be a little bit of pre-work to draft the set of stakeholders on the campus who may use the MADSMP and try to work through like what value it would bring to them. There will be like an MADSMP technical update and then we want to have a discussion around what you found in the pre-work section and then there will be and sometimes not all the times like some to-dos to take away from. So identify and upload five older DMPs into the MADSMP system to test it out and to test the functionality. So that's kind of the structure of all of the pilot program as we're seeing it kind of at a larger scale. I'm happy to turn it over to Q&A and to answer some of these other questions that have come in. I am going to stop the recording at this point though.