 Welcome, everybody. We will be starting in about one minute. Welcome, everyone. We will be starting in a bit under a minute. Thank you all for joining us, and we will be starting very shortly. Right. I think it's about time to get started. Let me welcome you all to the CNI virtual 2020 spring member meeting. And you have landed on the project briefing perspectives on digital scholarship programs. I'm Cliff Lynch, the director of CNI, and let me say how glad I am that you're able to join us for this presentation. Our presenter, I'm just delighted to be able to welcome back is Joan Lippincott, the associate director emerita of CNI, and one of the great experts on the topic at hand. And she's agreed to share a synthesis of her thinking and all of her years of work on this subject with us. Joan is going to talk. At the end she will take questions and Beth Seacrest will be moderating the question and answer part of the session. The best thing to do would be to put your questions as they occur to you in the using the question and answer tool at the bottom of your screen. And you can cue those up at any point. Joan will get to them at the end of her talk. And with that, I think I will just say, Joan, it's so good to have you back and thank you for doing this and over to you. Thank you, Cliff, for the warm welcome. And I'm just so delighted to be back with the CNI family and look forward to our communication this afternoon. Thank you for joining me in such a difficult period with this coronavirus pandemic. It is a trying time for all of us. And I hope all of you and your families and friends and colleagues are doing well. So I'm going to be talking about really taking the broad view about digital scholarship programs. As I was nearing retirement at the end of December in 2019, I realized that while I had talked about digital scholarship programs at many campuses, we hadn't done an overview of how my thinking had evolved at a CNI meeting. And so I gave this presentation or one very similar at the December CNI meeting in 2019, but it was not recorded. And so I asked Cliff if I could be part of this webinar series so that I could reach more of you and we would also have a recording of the presentation. I'm actually going to pause twice during the presentation to see if you have questions or comments that you'd like to make. And then we'll have a third and final Q&A at the end. I did want to start out by making sure all of you are aware that the CNI website has a wealth of resources about digital scholarship. There are videos of some sessions from our meetings, their voiceover PowerPoints, their publications from both Cliff and myself on this topic and many other resources. So please take a look and delve in more deeply to any topic related to digital scholarship that you would like to explore further. I want to talk about the framework that I use for discussing digital scholarship programs. To me, this is really important. Digital scholarship programs bring together new forms of scholarly communication. That's the SC at the top there. But they also involve teaching and learning. Now they may involve teaching and learning in terms of the use of the products of digital scholarship in the curriculum and or they could include students developing digital scholarship projects as part of their coursework. Of course, digital scholarship involves using technology and tools, but importantly, it also involves having people with expertise to guide students, faculty and staff in the use of those technologies and tools. And then finally, digital scholarship programs include spaces both virtual and physical. And you may ask our physical spaces necessary now right now when so many libraries and entire campuses are closed for many months. And my answer is that I do believe physical spaces are important. And I'll talk about why in one of the latter segments of the presentation today. However, there's plenty of work that can be done digitally and virtually while those spaces are unavailable. One of the things that honestly I find confusing is that many people don't understand why libraries are involved in digital scholarship. And my presentation today is going to focus primarily on digital scholarship programs in libraries, not to the exclusion of other types, but that will be the main focus. But in libraries, they've always had the mission to support research and scholarship. So I just can't understand why they wouldn't support a research the term usually used in reference to the sciences and digital scholarship a term usually used referring to social sciences and humanities. Libraries have been a venue that brings together expensive technologies and other kinds of resources for use by all campus departments. It's a level playing field for all units so that even if the sciences can afford their own expensive technologies, anyone on from any department on campus can come and use the technologies in the library. And importantly, they can always have also have access to the expertise that's available. Because say if there's a maker space in a college of engineering, even if they let others in to use the hardware there, which generally is not the case, but even if they did, they're generally not there to teach students in their off hours how to use the equipment and how to become a maker, for example. And another thing that I think is really important is that it concerns me that many students, whether they're graduate students or undergraduate students are finishing their degrees without learning about high end digital tools in their discipline that they could employ in future careers, whether it's in academic work or in a professional capacity. There are many faculty who either aren't interested in teaching students how to use digital tools, or they don't know how to do it and aren't and aren't making the effort to find ways to work with partners to make sure their students get that knowledge. And so libraries and information technology units are developing workshops, and other kinds of educational programs, including credit courses sometimes that are available to graduate and undergraduate students so that they can obtain these digital skills while they're at the university and emerge much better ready for the workplace. So the libraries space and programs are changing in concert with the changes at the institution but at their core libraries are still supporting the higher education mission. The three legged stool the research teaching and learning and community service community service many digital scholarship projects at universities involved some connection to the community though often the geographic area or segments of the local population. In addition, these projects support the kinds of societal values and principles that librarians have held dear for many generations, for example, equitable access to information, the stewardship of and celebration of cultural heritage. As we know, cultural heritage is increasingly being generated in the digital environment, and we need to know how to steward that and how to develop projects related to that. And digital scholarship programs can foster community, both in their physical space and in their virtual space. Some of our libraries reflect these kinds of things in their mission statements of their digital scholarship programs. For example, at University of Virginia, they say we're part of the library's mission to provide equitable access to learning to everyone. And while many people think about digital scholarship programs at large research universities, there are very robust programs at many liberal arts colleges such as Bucknell, a small liberal arts college in Pennsylvania which holds a digital scholarship conference for the past number of years. There are fewer digital scholarship programs from from my experience in the mid-sized master's level universities, but I'm sure that we'll be seeing more and more of those programs emerge in such institutions. So in my presentation today, I have a lot to cover. And I'm going to cover all of the sub topics on the left of this slide. So let's get going. In my experience, I found that when I hear the story of a digital scholarship program, many of them have developed rather serendipitously. Often it's because a faculty member wants to develop a project. He or she wants to answer new questions and can't do that without using new digital tools or large corpora of texts or other data. And needs some facility with some new tools and also needs the information in digital form in order to accomplish that research. And they reach out and finds a library staff member who's interested in working as part of a project team. And so off the program starts and develops from there. And often it's spread by word of mouth and another faculty member comes in or another graduate student and the program grows. As it grows, it does often focus on project work, but increasingly they may offer things like workshops and have guest speakers in so that people in the local academic community can hear about the range of types of digital scholarship and understand what's going on at other institutions. And so these programs may or may not develop as an outcome of a planning process. So one of the things that interests me today is we have a lot of participants and I'm really, really delighted about that. And I don't know what kinds of programs you might represent. So Beth, could you put up the poll that will I want to invite our participants to give me an idea of what type of digital scholarship program. What are they affiliated with. Let's see if that poll will come up. And if not, we'll move on in just a second. Okay, maybe that will maybe you can tell me in the Q&A or the chat, whether you're just starting out in your planning process, whether you've had a program for say up to two years, or whether you have a program that's been going on for at three years, those were the three choices. So, I want to talk about the program's mission and goals. And I think that it's really, really important for those involved in establishing a program, or generating a program. Oops, I'm sorry. I'm sorry, let me go back. I've got my Q&A. Oh, here we go. Okay, sorry about that. So, I think it's really important for people involved with a digital scholarship program to be able to really clearly and succinctly articulate a core rationale for the program. I think it's important to do that, so that you can explain it to administrators, whether it's administrator of the library or someone like a provost or department chair. But it's also really important to be able to articulate the rationale for library staff, because there are often people in the library who really don't understand what this is about and why the library is involved in this work. Ideally, and in fact to me it's key, that that rationale be tied to institutional goals and priorities. And therefore, I would suggest that it's important to, I think that it's important to understand who develops the program mission and the goals. Are you involving faculty? Are you involving people from other parts of the institution like IT or the Center for Teaching and Learning? Or is this just developed internally? Not every digital scholarship program has a mission statement. And, but you'll find some really good ones such as this one from the curve at the Georgia State University Library. And they say that it's a discovery rich, it's a technology rich discovery space supporting research and digital scholarship for students, faculty and staff. And they want to enhance research and visualizations by providing technology and spaces that promote interdisciplinary engagement, collaborative investigation and innovative inquiry. And I would be very, I think it's very likely that those last three terms relate directly to things that are from the institutional priorities or mission. The statement from on digital scholarship at Columbia is really interesting because it so reflects the core values and principles of libraries. So they are committed to collaborating with students and faculty in the creation of new modes of knowledge creation. They believe in crediting labor, in foregrounding accessibility, providing equitable access, actively listening to traditionally underrepresented voices, etc. So a lot of key things that represent core values and of both the profession of librarianship and mission of the university can be reflected in these mission statements. When I visit campuses, I find that many people have the idea that that digital scholarship programs are actually digital humanities programs or maybe digital humanities and arts programs. But in actuality, many, many of these programs support not only the humanities and arts but social sciences and in some cases the sciences as well. Now there's no right or wrong here, it's just that this is something that needs to be discussed and planned for as you develop your program. And then who is supported? Generally, probably all of these programs support faculty and researchers, most support graduate students, and some also support undergraduates, particularly if they're working on things like capstone projects. And what's the range of offerings? If your digital scholarship program is just starting up, but you've had a GIS program in place for years, how do they relate to each other? Do they merge? Are they part of each other or do they stay separate? What about a data visualization space or a maker space? And how does this relate to other campus facilities or programs or other library units? I'm going to spend more time on this range of offerings and the relationships later in my presentation. So here's a statement from the University of Houston Digital Research Commons. They say they work with faculty and students in the humanities, the social sciences and the experimental sciences. And at the UVA library, they're working on experimental scholarship in all fields. Whereas in the digital lab at the University of Georgia, they work on digital humanities and it's a joint partnership between a campus humanities center and the library. Ohio State University's Research Commons does one of the best jobs that I've seen in exploring and explaining the campus relationships to other units that work with research and with data and with high end technologies. This is just a part of their page on these relationships. When I worked with my co-author Quinn Dombrowski on an e-car paper on supporting digital humanities, we felt that one of the biggest concerns was that faculty and students had great difficulty in finding out where they went for what kind of support on campus and having something that spells all this out in one place can be very beneficial to your institutional community. Who administers and governs these digital scholarship programs? Many of the programs that I'm describing here today are administered solely by the library. But there are also quite a few that have the library plus a partner or partners who administer those partners could be the central IT or academic technologies. It could be a teaching and learning center or a or it could be another area of the university such as the research office or the graduate school office or one of the colleges or a college or department or institute could administer the program without the library as a partner. It's difficult to find any information on governance or advisory groups for these programs in asking about this when I visit campuses. I often find that it seems that the oversight is through the library faculty committee, the kind that's often appointed by the faculty senate that deals with any type of library issue in the policy sense. So here in terms of administering the digital scholarship program, the digital scholarship commons at Northeastern University is jointly administered by academic technology services, the center for advancing teaching and learning and the digital scholarship group of the library. And in terms of oversight at Yale, this is one of the few that I've found. They have an advisory board that is part of that is a function of an overall campus digital humanities executive committee. And it's really interesting to see the varieties of sectors of the campus that are represented on that advisory board and there are more names that I couldn't fit onto this screen capture. One of the things that I've been working on for some time is to try to delineate different types, different models of digital scholarship programs or digital scholarship centers or labs. And I've chosen to label these for the purposes of drawing the distinctions as either a center or an institute. Those names are really somewhat arbitrary, but they help me just organize the characteristics. So the way that I see it and not everyone agrees with me on this is that a center is library administered or primarily by the library with some part possibly with some partners. That means the library is responsible for the staffing for the overall budget, etc. And the primary funding is from the institutional budget. It may not be just from the library budget, but it's from perhaps some money from the provost's office or the research office or something like that, but internal funding. And one of the strengths of these programs is to offer both the expertise, the kinds of consultation and working with projects, as well as education to a variety of disciplines and users. And then there's a strong interest in life cycle issues. What do I mean by that? It means that they advise on such things as metadata creation for digital projects, on digital preservation, on publication, and what that means for digital projects, etc. And I contrast that with an institute, which I consider is faculty or academic department administered. And the people that are part of the institute, they're usually affiliated faculty whose primary funding for their digital projects is from project grants, whether they're external or internal, often in a defined discipline. And this is where we see a lot of digital humanities institutes. And their main raison d'etre is to answer new research questions, very research focused, not so focused on life cycle issues. Now it is a continuum, and I agree that it's not the only way to view these, but I think that it can be helpful in understanding what type of center you want to create, or when you visit other centers to understand what they're doing, why they're doing it, and whether you want to develop your own program in one model or another. Now when I said there's a continuum, there are a number of programs that are jointly library and faculty administered, often with dual heads. But in those programs, there can be a real distinction. Usually the faculty heads the research initiatives of the program and the library lead may be primarily someone who schedules staff and is overall responsible, say for the budget or for making sure the equipment is working that kind of thing. In other programs, the faculty is the head of research, but the library is the lead for programs. By that I mean things like the consultation programs and the educational programs. And it's really important if you're developing these kinds of jointly administered programs to understand what is the nature of the partnership and collaboration is one party in a sense subservient or less important than the others. Do they have an equal say of who establishes priorities and who allocates funding. So I think all of these are critical issues. So I think this is an area ripe for discussion. And so I'm going to pause here to see if any of you have questions for me at this point or comments to make and So, Beth, I'll ask you if there are any questions that I should address from the chat or the Q&A. Please continue to type them in if you would like. Yes. And my apologies for the poll. There's something so I hope you'll try the second one because I'm not sure exactly what happened. But Stephen Bell is asking if there are any digital scholarship programs left that are run and stacked entirely by academic librarians. It would seem by now some stuff would be non library professionals or there'd be some faculty involvement at some level. That's Steve. I would say that's correct. There is almost always a variety of staff involved and I'll go into some of those types of staff a little bit later. But what I would say is that faculty are not always involved as staffing of the centers. I know of a number that don't have faculty who's let's say that part of their salary is allocated to the digital scholarship program. On the other hand, there are usually a range of types of staff in the program. I hope that answers your question or your comment. Anything else, Beth? I don't have anything else. Thank you. Thank you. So everybody wants to know about funding and I don't have particularly good news. I don't think because in general the types of digital scholarship programs that are housed in libraries. The primary funding comes from reallocation of the library budget. This means reallocating positions to the digital scholarship program and reallocating other resources. Now other parts of the organization may chip in. IT may provide equipment and may provide staff. A research office or a provost office may contribute. In the past, I don't know when we'll see this again given the belt tightening that we're anticipating in the coming year or more. In the past, some provosts have made cluster hires of faculty who are involved in digital projects. And I know of at least one case in which one of those positions was allocated to the digital scholarship program in the library. A college or a department might contribute funding to the program as well or may host if they run the program if they administer it, of course, then they're funding it. Some funding may come from cost recovery, but I don't know of any program that is primarily funded that way. And of course there may be grants, both grants to the program of which there aren't a lot or grants for specific projects. I think that for programs that have been in existence for say two plus years, we're going to see more work on setting tiers of service because the programs just get more and more popular. And the staff start to feel a bit overwhelmed by the demand. And so I think we need to understand better how to set tiers of service and it would be great to have some models of organizations that are doing that. The one that I know has done this to some in a certain way is the University of Victoria library, which has a very nice resource that shows what they will do for free and what they will start charging for in terms of digital scholarship projects. We had a project briefing on that at a previous CNI meeting. And what is cost free to the users versus what is charged? For example, if someone wants a large collection digitized in order to do their project, will the library charge for doing that work? Will the library request that when a faculty member prepares a grant proposal that they allocate some portion of funds to part of staff members time or to other kinds of work that might be done by the library. So these are all possibilities. And these are the types of staff and I'm sure there are other types of staff that are commonly in digital scholarship programs, librarians, subject specialists, tool specialists, programmers or coders, departmental faculty, information technologists and I include their people like instructional technologists, instructional designers. And I would also include, as I was reviewing this, I thought, oh, I didn't even talk about designers or user experience people and there's so many types of staffing that can be involved. And when we did our very first meeting about digital scholarship programs, a specialized meeting at CNI, we found that almost every program employed both graduate students and undergraduate students as part of their staff. So most of these programs have quite a variety. And it's important to understand what skill sets are needed and of course those skill sets are going to change as new software and hardware emerge and as new needs surface from the faculty. Importantly, who can be retrained both the existing staff of the digital scholarship program, but I think more importantly, the incorporation of staff throughout the library organization. Again, if libraries aren't going to be able to fill positions in the near future, how can they develop the existing staff to become more involved and is there redundancy. Many program staff have told me that they're very concerned that if someone leaves the organization, the digital scholarship staff, and they're the only person that knows a particular kind of coding, what happens to the project that they're in the middle of. These are real issues and real problems that need to be thought through. One of the trends that I see are hiring recent PhDs who've worked on digital projects to become either heads or staff of digital scholarship programs in libraries. And an issue that I think needs to be addressed is how much of that individual's time may be spent on his or her own research versus the institutional priorities. By that I mean consulting for other projects or providing workshops or developing a speaker series, things like that. I think as long as those expectations are well spelled out, that is fine, but I think there are some instances where someone may come in thinking I'm going to do my own research and maybe on the side answer a few questions on other people's projects. Now a different way of looking at this is that some digital scholarship labs or programs actually develop software as one of their outputs. They do that as an institutional priority and some of them work on projects often related either to the university or to the local community and they produce a digital scholarship project as part of their own work. If there are some administrators as participants today in this webinar, I hope you'll think seriously about talking to your digital scholarship staff about how to approach faculty at the beginning of working with them on their projects about how they might be credited when those projects are completed or put out for the public. So what will the staff be listed on the project site and will their contribution be noted? Can it be negotiated in advance whether the staff or some of the staff may be able to co-present at conferences or co-author any outputs of this project? This is really important and isn't getting quite enough attention, but this should be thought about at the very beginning of the project and not wait till the end when the faculty members already signed up to do a presentation on his or her own and the other team members are excluded and not adequately recognized. I really enjoy the websites of programs that show their staff and tell a little bit about what they do. Here's a really nice one. This is just part of it from Dartmouth and you'll see the variety of departments that those individuals come from. And at University of Rochester they highlight the specializations what the particular staff work on and what they do. This is really useful for getting the message out to your broader community of the kinds of expertise that are in libraries today that many people don't realize on your own campuses exist. So when I talk about a program, what is in a program? Well, these are some of the things that I think about. Project consultation is a big, big piece of what many digital scholarship programs do and it can be all types of project consultation. And this is where we see this involving people from other parts of the organization becomes a question. How do you involve the metadata specialists or the preservation specialists in your library, in your digital scholarship program? In addition, almost all programs offer some kind of educational program, workshops, software carpentries, certificate programs, even credit courses in some cases, and sometimes fellowship programs. And almost all offer a speaker series and they also work on creating a peer community and a physical space can be really important in helping to foster that. In this first meeting, that specialized meeting that we did at CNI about digital scholarship programs, the programs represented said that they felt one of the most important things they did was help to create a peer community on campus of people working on these projects. There's a lot of peer learning that goes on in digital scholarship programs. There is no simple way of deciding what is or is not included in the digital scholarship program, GIS, data visualization, scholarly communications, and on and on. I would love to see more synergies between digital scholarship programs and special collections and archives. I could see much more work being done with information literacy involving specialized special collections and digital special collections and have working with faculty and students to develop new digital projects as part of the curriculum, for example. There are many synergies between and among all of these different types of programs that may be in the library or may be elsewhere in other units on your campus. And again, representing what kinds of services and spaces and equipment that are offered by the program is really important to do on the web. Here is one from Brown University Center for Digital Scholarship. And here at Lafayette College, another liberal arts college in Pennsylvania, they say their team can help you manage research data, use and build digital collections, create visualizations of your research, generate projects using geospatial data, and design digital humanities curricula. So spelling this out for people I think is really important and a way of communicating what you have to offer in your program. So I'm going to pause again for questions and see if we have any that you'd like me to address at this point. Joan, Dale Aske is asking, have you seen any institutions where they are on a distinct second generation of such a center? In other words, have we seen any wind down with a subsequent relaunch? I certainly have seen programs that are in the second or maybe third generation and I would say Brown and UVA would probably be right up there on that list. I might be able to think of others, whether they had a formal relaunch or not. Sometimes it was a matter of a reorganization or who was responsible administratively for different programs, etc. But I would probably look there first for examples. Those are really long standing programs. Okay, and that is all I have for you. Beth put up the poll the last time and I didn't see it. I don't know if you out there saw it. It was on her screen and we're going to take the second and last poll now because I'm interested in. I'm going to move into the segment on physical spaces and I'd like to know about your campus and behold, there is the poll. So if you would answer this poll, I would appreciate it. I'd just be curious about what physical facilities you have available. I don't mean today. I know you're closed, but the hypothetically are available. Whether you have some but not in one designated area, whether you have a purpose built design and configured space with multiple capabilities or whether you have a variety. So it says host and panelists can't vote. So I'm not sure what that means. I just saw that at that bottom. So we're going to move on then I'm going to close that. And we're just going to move on to spaces. I'm keeping this list of adding to it of what digital scholarship spaces can include. I'm not going to read through these, but if you have any for me to add, I would love to have more ideas of what, what to look the poll worked. Now it's, we're sharing results and we have a real mix and 30% across the board very, very interesting. Thank you for responding. Okay, I'm going to close that. So the Florida Institute of Technology, I thought does a particularly nice job of showing their facilities on their website. And it's something that will help market those spaces and spaces for consultation and project management I think are really the basic, you know, if you're going to have something you need that. It doesn't have to be a designated space. It can be someone's office or a group study room that's designated and can be reserved for project consultation. It and it doesn't have to be fancy, but this is a nice space at the digital lab, where you can see people could easily sit in the easy chairs and just start a discussion, or they could move to the tables with a large screen on the wall, and and discuss their project or the project management and then look at the project as it develops over time. We need places to do the work of digital scholarship. And by that I mean both providing access to things like large screens or dual monitors to specialized hardware and software that people may not have on their own computers or in their own departments. And importantly, like here at the DH lab at Yale, access to the expertise to work with the user community. At Duke University, the Edge has some really wonderful spaces. It's a very large facility and these are just two of the areas. The photo on the left shows some project rooms, and you might wonder what the stools are in the hallway. Well to the left of the stools is one really long whiteboard, and you can envision that being used say to create a project timeline or to do brainstorming for a project. On the right, you can see one of their specialized labs with the dual screen monitors. And one of the things I particularly like about that are that they're displaying some of the outputs of the use of some of the software that they make available. At the curve at Georgia State. These are group workstations where they have access to specialized software and students use these in teams after a class session. At McMaster and Dale, this is an old slide of I took while I was there a number of years ago, but those tables and chairs on wheels can be easily reconfigured so people can work in teams during and hands on workshop. And at Brown University, they develop this space outside of their digital scholarship lab which has a large visualization screen so that students can continue to work on high end projects after they've had a class in the classroom space. At the University of Calgary the they created lab next as the direct result of a melon funded grant project examining the needs of multidisciplinary research projects. And I was part of the external review team for that project, and it was really interesting how the faculty involved in those projects commented how important lab next was to them to develop their projects they had no other place on campus where they could bring those interdisciplinary and multidisciplinary project teams together, and that it was critical to their achievement of their results. When I visited University of Minnesota they just pulled some chairs together in a corner of their digital scholarship facility and gave us short presentation using the whiteboard and a portable screen and then we toured the facility, which included a display of some of the projects done by their faculty there. At NC State at the Hunt Library, they've reconfigured part of their space to a new area called data space, and you can see on the right that it's a place where people can drop in for consultation on such software as MATLAB or Python. When people ask me how important digital scholarship physical spaces are one example I would use our large scale visualization spaces you can't do this kind of scale virtually and so but not everybody wants or needs to do a large scale visualization space, but if you do then this is the kind of fiscal space that you might want to to visit and to think about including in your own facility, or a large display like the one at Georgia State this is where I said people can attend a workshop and they can add rows of chairs, and then after the workshop they can disperse to those team workstations. At the University of Maryland myth center I attended their 20th anniversary celebration and at the rear of the photo they had held presentations and then at the closer side of the photo closer to you, you can see some of their staff workspace and to the left there was a reception area with a small kitchen. At Yukon at the greenhouse studio, they created something they call the kitchen table to promote a sense of community and to be a place for brainstorming or just talking informally about projects. And a kitchen area I think is a great way to establish community or promote community coffee breaks or lunch bring people together for informal discussion. It's also important to have mechanisms for communicating information about the new forms of research and NC State just does an unparalleled job of both talking about the technologies and expertise available but also about the kinds of outputs that people in their community produce. So I'm going to move on to my last substantive topic before some final remarks and that is assessment and this is going to be brief because there isn't a lot to talk about here. There are some very good needs assessments out there, and usually those include environmental scans including of the local institution, you know different parts of the university or college. Usually there is either a survey or focus groups or interviews or all three that address faculty and student needs, and then the report will talk about the gaps between what exists and what the needs are and then address the existing staff capabilities. In terms of program assessment, what are the measures of success? I don't know if you do 20 workshops in a year or in a semester is that good or is that not good. If you have 10 attendees is that good or is that not good. Have you achieved a goal? Have you reached the intended audience? Have you reached important constituencies that are important to your institution? So I really think that understanding what your goals are and then developing quantitative and qualitative measures related to those are really important and are underutilized at present. Now it doesn't mean that everything has to, every program development has to be matched to those goals because unanticipated opportunities come up and they may be the best project of the year for a particular program. And it's also important as you think about assessment to think, when you finish an assessment, what are you going to need to communicate and to whom? To a library administrator, to a provost, to some other group, you want to talk about your achievements, but also the shortcomings and what you need to move to the next level. So here are two good examples of needs assessments, very thorough, the one at University of Pittsburgh and one from Harvard and full disclosure, the one at Harvard was done by my daughter Sarah Lippincott. The University of Houston also has a very good needs assessment, but we don't see enough linking the outcomes to the goals and priorities of programs. And that can be really, really important for stepping back and recalibrating a program. So I'm leading almost to the end here, and I want to talk about opportunities and challenges. For me, digital scholarship programs are such a great opportunity for libraries or other units on campus to contribute to the college and university mission and goals to address core research, teaching and learning functions in the digital age. It's as simple as that, and it's as important as that. But there's so many challenges and they may be different in different institutions. I've seen, for example, a number of digital scholarship programs, often digital humanities programs that are in libraries, physically located, but have no relationship to the library staff or the library program. I see in other cases, staff stretched thin because they've really been recognized by people in their institution as having expertise that's very needed and very valued. But without establishing clear priorities or having ways to prioritize requests for resources, it creates a lot of stress. I think one of the answers to that is incorporating more non-center staff. Now, while many programs are stretched thin, it's also the case that there are many in the university or college that are unaware of the expertise and technologies available in the library. And of course, assessment and recalibrating programs are big challenges. So if I were attending this webinar, I would like to come out of it with some thoughts about what kind of program do I have and what kind of program do I want to create or what kind of program do I want to evolve to. What will constitute success or what does constitute it? Is the staff ready? And are there parameters? If you're going to be working with other units or other services, do you have clear parameters for what those services and partnerships will look like? I also have some key questions for the overall CNI community. The important one for me is how do we transition to digital scholarship support as a norm and not considered something special or other? How can we involve more staff in digital scholarship activities? I think those first two are related. And how can we work together as a community on some core problems? For example, one is that faculty often come to the library having created a database in their area of specialization, but they have no idea how to preserve it. And it may be in a state that the library doesn't know how to preserve it either. So how can we work on a core problem like that to come up with some solutions for the future? So now we have time. If you have some questions at the end, I've left just a little bit of time, but I also want to say that I am still using my CNI email address. So if you want to contact me after this session, you can send me an email and I also have my webpage with publications and presentations is still up on the CNI website. So thank you so very, very much for your presence today. And Beth, over to you to see if we have some more questions. Thank you so much, Joan. I have a couple of questions from Steven Bell. His first one, not quite digital scholarship, but are you seeing maker spaces becoming a common feature of the digital scholarship center as well as activities like gaming? In terms of the maker space, that was one I honestly have had a little difficulty understanding the relationship between maker spaces and digital scholarship, but I'm beginning to wrap my own brain around it a little bit more. I think that there will be some issues depending on the type of maker space in terms of if their activities, for example, that involve fabrication, you'll want areas that can be segmented off so that people needing a quieter environment won't be disturbed, you may need, you know, special configurations for other types of equipment in a maker space. I'm also not clear how much the skill set of people who staff maker spaces overlaps with the skill set of people working on digital scholarship projects, and I would love to be enlightened on that. So if some of you either want to comment or send me an email about that, I would very much appreciate it. In terms of gaming, game development, certainly I would see game development, for example, as a potential product of digital scholarship. I could think of games using, for example, images from digital images from archives or special collections or developing new art. forms or incorporating new types of music. Another thing with developing games is understanding the intellectual property issues related to what's included in the game as well as then how you want to share that game after it becomes finalized or becomes a product. And similarly related from Stephen and he says in your travels to these DSC's how many have programs geared to undergrads to get them connected to digital scholarship seems most of the work is geared to traditional scholars, but are we missing an opportunity by overlooking upper level undergrads. Well, Stephen you know me well and know that I very, very much favor working with upper level undergraduates and yes I do see it as a missed opportunity in some cases. However, we have the example of liberal arts colleges, and probably all of the programs in those colleges that I'm aware of absolutely work with undergraduate students so they are a great model for the larger research universities. Now again some may say well but you have to prioritize how are you going to serve all audiences, but one of the things that I would say is that working with those undergraduates can mean that they become staff in your digital scholarship programs and you know, a mechanism for refreshing skills and for incorporating new skill sets, not just in terms of their own education but in adding to the program of the of this center itself. And that's all I have in that Q&A. I want to thank Joan so much for her presentation this afternoon we finally have it on video and I am posting in the chat a link to our meeting I hope you will sign up for more webinars and thank you all so much for attending today and thank you again Joan for your It's great to have had all of you here for this presentation I really appreciate being continuing to be part of the CNI community. Thank you Beth, thank you Cliff and thank you all of the attendees. Thanks Joan for a wonderful talk and you're always welcome. Thanks. Everyone stay healthy and stay safe. I wish you all the best.