 Thank you for joining us and welcome everyone. I'm Cliff Lynch, the director of CNI. And you've arrived at one of the project briefing sessions in the third week of our virtual spring 2020 member meeting. Today, we have a presentation on a really interesting and important topic, which is how to deal with experiential research and scholarship. The entire sort of chain of managing that kind of material. We have two speakers, Mika van de Grift and Shelby Holman, both from the North Carolina State University Libraries, and they'll be, they'll give us a presentation. Then we'll take questions and answers at the end. These questions occur to you during the presentation. There is a Q&A tool at the bottom of your screen and please feel free to enter questions at any time as they occur to you during the presentation. When our two speakers are finished Diane Goldenberg Hart from CNI will materialize and moderate the Q&A session. And with that, let me just thank our presenters and thank you for joining us. And I believe Shelby is going to lead off so over to you. Thank you Cliff. Hi everyone thank you for joining us remotely today. Mike and I are going to talk about the immersive scholar grant and particularly look at development documentation display and dissemination of experiential research and scholarship. We have a lot of project project documentation through our website at immersive scholar.org and Mika has shared that link in the chat. And we also have a repository of the different documentation and project outputs within an OSF repo. It's also accessible through the link. So this presentation is going to be split into two parts. I'm going to give a pretty quick overview of the grant some of the outcomes and lessons that we've learned and then Mike is going to focus on the theoretical frameworks and practices that we applied. So to give some context what is immersive scholar. Since the Hunt Library opened in 2013 we've been able to move beyond the initial installation and adoption of our high tech environments such as our teaching and this lab shown here to think more broadly about the potential of these immersive environments. This led us to the realization that the utilization scale and impact of visualization environments and the scholarship created within them is limited due to a number of factors. We identified this problem in 2016 the library's first proposed and received a grant specifically focused on addressing this problem statement, which led to the visualizing digital scholarship and libraries and learning spaces grant, also known as immersive scholar. This is through the Andrew W. Mellon Foundation. It's a three year grant. We are in the third year currently, and it's $414,000 and focuses on two key areas. One is developing a community of practice. Hunt Library is not alone in implementing custom technology. Every institution utilizes different programs and approaches to maintaining and using their environments with most institutions operating independently, which is partially due to the relative newness of hosting such technologies. And so we asked the question how can we connect with other groups and share knowledge. I also wanted to look at technical and resource barriers that limit participation and impact. Since each display is unique, the program and support must be individually crafted, requiring each institution to employ experts in specific technical skills, or other areas to maintain these walls. Additionally, funding for both the maintenance and content creation for these environments is expensive. We're pretty lucky at NC State. We have some really great creative students and researchers that help us create custom content. But every piece is really time consuming and requires really particular skill sets. And so that leads to limited content being created. To adjust these two areas, the grant was split into five main components. First was gathering together a group of experts within visualization to form an advisory panel. Second was funding sub grants at five different institutions, which I'll discuss in a second. We hosted a goal setting workshop in December of 2017 to determine priorities for the overall grant and for the sub grants. We also hosted three creative residencies at NC State, which I'll also touch on in a minute. And then finally, we hosted the Liberac 2019 conference to pull together the broader community discuss work that will continue in this field. So what did we accomplish with this project. This project is multi layered in terms of the project products and principles. It's also multi institutional and multifaceted. So there were quite a few outputs and so I'm not going to go into detail about any of them, but to just give a sense. First the sub grant projects. So five different institutions completed projects with the goal of either lowering the barriers to access or building a community of practice. This was financially the largest part of the grant. There are five details available at our website immersive scholar org slash tools, and to give two examples at Indiana University they created the collect them, which is a content sharing platform to facilitate the use of tile video walls or similar large format displays. So for example is at VCU they focused on creating a suite of tools for institutions that are looking to formalize their own spaces and services, such as a best practices for local needs assessment and conducting external visits. And then, in addition to managing the grant NC State's largest contribution was hosting three creative residencies. Each of these had the goal of sharing the work that was created openly with interested institutions, and every residency focused on creating digital data driven pieces for immersive or large scale environments. So for instance, some examples include coated glass by list of floor that created digital stained glass pieces based on me to march signs. And then there's a look at SWIC which are generative gardens based on data from a housing and food and security survey given to NC State students and surface tension by Caitlin Amisha that used us GS data to investigate the human interaction with water. And so in addition to the outputs from the residencies and from the sub grants, we also had a number of overarching project outputs, which include testing frameworks recommendations for describing scholarship to help with evaluating giving credit to the scholarship that's created in these spaces, and also guidance on hosting your own experiential scholarship projects. All of these are available through our website or through the OSF. So I'm going to give you a quick guide gears a little bit to think about how we accomplished all these different components within this three year time. We employed a number of different approaches and two that I want to highlight really quickly. The first was to use a team based approach. What we did was integrate a core management team. So on this, this screen, we have the five current personnel that are involved with the core management team, as well as past contributors and advisors. And really think about the overall timeline, making sure that we are hitting the goals, thinking longer term and also managing the components and mix that with functional teams to support specific projects. This image is from one of our residencies. And so you can see there are four different groups that we're helping to support the artist during her time here in creating her work. So this helped encourage agility and lesson time demand on specific individuals. So we put together flexibility with rigidity with so many concurrently running components we had to set milestones to stay on track, but we also need to be flexible and how those milestones were accomplished. And we found that by doing this it really encouraged a lot of creativity and offered up new forms of experimenting with the scholarship or the way that we can work within these technologies. So we started a few lessons over these past three years and to limit it to the top three lessons that we've learned. The first is considering sustainability. During this project we had a few staff that either changed roles or shifted positions and so it made it really necessary to think about what is the sustainability of managing a project like this with so many components at the same time. When the grant went on we shifted the management approach to try and lessen the time burden on contributing staff and to ensure that the project lead and the project manager had dedicated time to be able to work on the grant. So this helped and it actually is helping a lot right now too as we are adapting currently and extending the grant a little bit to be able to accomplish the final components. We also thought about and learned a lot about considering sustainability surrounding content. There are multiple creators and owners in this situation. And so with so many different projects and content types. We had to ask a lot of questions about who maintains these products over time, especially using this technology. And we learned to consider the means of displaying this content. These environments and the scholarship created within are really unique experiences and how can we capture that, especially once the technology needs to be refreshed or starts to fail or is eventually phased out. And so we don't have a firm solution to that, but we have learned to really think about these questions and try and build them into the guidelines or workflows that we established at the beginning of a project. We also learned to consider scalability early and often at the heart of this grant is sharing content amongst different partners. And so we had to think, how do you go from displaying something at one location to displaying it at multiple locations. And it's really important to think about what does that require. What are the dependencies what components are necessary and thinking through will other institutions have access to the same software or expertise to put these things in place. And having a lot of conversations with partners, thinking through how we are workflows work in terms of implementing these content at our university, let alone another university. We also learned to consider scalability when it comes to resource allocation. For instance, that example from earlier with the functional team that translated in 13 staff that were actively involved in the project and that's not counting other staff members who may be came and supported through their components in smaller ways. And so if we were to grow the number of residencies it's not possible to dedicate enough staff to do that really quickly. And so we had to learn to do really thoughtful project scope and intentional growth to ensure longer term contribution. And we actually implemented this through two local residencies that we hosted separate from the project but with the same team to look at how could we continue this on beyond the timeline of the grant. And finally, we learned to really think thoughtfully about implementing guidelines and workflows. So not only for the team, in terms of staying on track and meeting our markers for success, but also think about the products that came out, making sure that individual sub grants are sharing common expectations to help facilitate that sharing and also thinking about evaluating such different outputs from the from residencies or from institutions to ensure that there is credit and evaluation. So with that, I'm going to turn it over to Micah. Yeah, thanks Shelby so and that's a great foundation and the background of the project almost to the point where we are right now. And I'll back up a little bit and say that throughout the project in the middle we started to look at some of the opportunities that we had since we were working in a sort of a broader space than we had originally thought. And so what you see on the screen is a reiteration of that that core principle that we kept going back to that was in the original narrative that through lowering the technical barriers to this kind of work we're out we're creating a space or the possibility to increase the impact of these environments, and but also the works that are created in them. So that that allowed us to sort of move into a new thought space where the form of work that we're working with exists almost entirely outside of what we had been what we have in libraries called scholarly communication for a long time. And certainly the journal and monograph text based scholars communicating to one another. And really what we were functioning in is a space that was much more akin to public scholarship or media arts. And so the phrase that we've started to use is this is communicated scholarship, not necessarily a scholarly communication in the way that we've traditionally thought of it. We're also thinking about what opportunities does that afford us in terms of rethinking digital scholarship overall as it's slowly grown in our field. And what we realized is that the kinds of works that we were creating are meant to be experienced that the hunt library, and also our cohort partners, immersive or large scale spaces or virtual reality as it was included through a kind of grew in impact in the course of our three years that that is that requires a physical environment so it is digital scholarship but we're also thinking in an analog or a situated or a physical space that is also predicated on the user and in person experience. So what's been helpful for us is to start thinking about borrowing many different bits from many corners of library work to make what we're calling this this patchwork quilt of good practices that are allowing us to think about how we want to increase the impact, not just of the environments but also of the scholarship created within them. So I'll go through four different corners of library land where we're borrowing some ideas from and then provide a couple of quick examples. First from open infrastructure, which is a conversation that's still active and developing. But we've what we've borrowed is this heightened sense that the services protocols standards and software should empower and invite communities to participate in the building, not just of the project and not just building but also maintenance of workflows and skill sets and policies and procedures that surround the work. So one way that we thought about that was that from the outsour from the outset of the project the cohort would be a part of building advising developing testing and utilizing everything that we all co created together. So we foregrounded the cohort as this community of practice and facilitated sharing throughout the project and are still finding new ways to do that as we're going toward the close. So we decided early on that the content and the tools and infrastructure are important parts of the project but we wanted to make sure also that documentation is thought of as a primary output from the grant. So that's why we have and you can see there at the link to our open science framework repository we're really thought deeply about documentation as as essential to the project. And in terms of delivery and dissemination we chose to deliver all the content except for one project that has a technical dependency on the web also for maximum responsiveness and accessibility so we're sort of trying to stake this good practice claim that even for situated physical experiential works and online component is also critical. So public in community engaged scholarship another another lens through which our broader community is working. We adopted this ethos of interpretation and translation as essential and central to the project. Another example of that is that our creative residents Caitlin and Misha worked very closely with our local USGS office on the preparation of surface tension the prototype of the project you see right there. So there were there just happened to be a water resources conference that happened during the time that they were here so they went and sort of had a table in the corner of the the Expo Center for that conference. Another example is that we have several living and learning communities in our on campus at NC State. So we tried to make sure that our residents were integrated into those living and learning communities, often through through a talk or a quick Q&A session. So Lucas Swick for example went and presented at the artist village. So we're trying to find ways to not just make things within the university but invite the community of the university and the community beyond the university to work with us in the idea space that we're building. From the area of principles based research production. We inherited this impulse to infuse our project with values of justice of valuing labor responsible stewardship. How this played out and she'll be mentioned a few of the projects already and we purposefully chose and crafted our call for proposals based on projects that were dealing with contemporary and important social issues like food and housing insecurity at the me to movement or water access. Coming out from our observance of what's happening in the digital humanities especially we thought it was very important to find ways to attribute the labor that's been involved in the project so we developed a contributorship data model and methodology, which we're now going back and refreshing and applying to each of the residency projects but also to the project overall. Who did what how were they involved finding ways to make sure to that it wasn't just the three PIs that that did this work there was a broad group of people and we can describe how and what they were involved in. And finally, I thinking back to the MIT future of libraries report. That's the concept of the library as a global open platform. We've actually begun to reimagine the physical building that James behind library as a container for these scholarly works much in the same way that a journal or a monograph or a platform are the containers that we're starting to understand as the delivery of scholarly works. The library itself I think can be one of those containers. And so what decisions that we've tried to make or how can we give these works in our library as many markers of that containerized knowledge as possible, but still say that the, the, the premier way to for you to engage with this work is in the building. And here's all the other ways that you can also engage with that work. So my ability was was something that was really important, which was a reason that we decided to use open science framework. So all all the products of the, the grant, including the projects the content themselves are our sightable with do is clearly indicated in online and scholarly venues as we share the work broadly. It's really helpful and useful for us to engage in peer review in some way so we submitted the immersive scholar project to the newly launched journal reviews in digital humanities and you can see the DIY for the review of our project that was published in early March. And this is what the thought space room right now is we're still thinking about what it means to publish what does it look like to publish these kinds of works. So we're in conversations with our colleagues at UNC press to discuss what, how do you even do this sort of thing what does it look like to publish an experiential works of scholarship that exists in the same way in a primarily text based system but is a very different kind of thing. And the score again is so Shelby and I sort of worked on our sections of the project separately and then and then put them together, and we both came up independently with the same challenge so I forgot this slide. This is basically a concept for what publishing workflow might look like for for these kinds of works, where it's created at one library, and then displayed, and then sort of shared out in this network or this cohort or this library to be displayed and shared in different ways, which creates a distance from the project. And then there's some sort of certification that happens maybe through University Press which applies some authority, and then a different library would say okay well that's something that's valuable to us. So this is a concept that we're developing out of the immersive scholar project. The challenges that that Shelby mentioned were the exact same ones that I came up with without kind of comparing notes until we're compiling the slides together. And they are they are the same how do we sustain this kind of work how do we sustain the technology but also how do we sustain new content. How do we allow for a femorality of these kinds of things where they don't need to exist forever but there should be a way to refer to the thing as it existed at a certain point in a certain place. We're thinking deeply and broadly about the shareability of these things yes it's great to have large walls and screens but what does it look like to scale down to a cell phone or to deliver it in a static web environment or low tech distribution. And then finally the, what is this the scholarship NIST of this of these things. The best example we have is that our, our, our colleague Mary Hasket, who did that that study about food and housing insecurity at NC State. She needed a certain degree of scholarship NIST for the work that she did, but Lucas Swick who is our artists who came to do a residency with us. You know he works as a developer and he's a code artist. For him citation impact isn't something that matters in the same way. So we have to think about balancing the impact needs of different kinds of people who come together to work on a single thing. To close I'll say that I think we've learned into underscore what Shelby said we've learned quite a bit about the development and display of this kind of work. And I'd say right now we have a really great baseline for how to document and describe the works themselves and the work that went into it the labor. We're experimenting and exploring what dissemination of these kind of works can look like that I think will be a little different than what we've understood to be digital scholarship projects or digital humanities works, or especially journal articles and data sets and other things that are being shared right now. So that that's the conclusion of the presentation and we're happy to hang around and take as many questions as you all have. And Diane has just put in the chat that you can put them in the Q&A box, or we're happy to take them in the chat over on the Zoom window, and I'll mute for a second. Thank you very much for listening. Thanks, Micah. Thanks Shelby. That was a fascinating talk really interesting topic. Lots to chew on there. Lots of interesting graphics and pictures to from your fabulous library. So thank you really appreciate that and I want to thank you again for coming to see and I and I want to welcome all of our attendees for joining us today. As Micah called your attention to the chat box, he put out some useful pointers for you to explore the immersive scholars website, which is a fascinating website has lots of great tools and resources there so I hope you'll check that out. And it looks like we have our first question in the Q&A so let me go ahead and read that to you. How much communicated scholarship experiential scholarship, the projects themselves not immersive scholar as a project, then received in RTP processes, you mentioned the citability but not necessarily the publication aspect. Shelby you mind if I take a stab first. The quick answer to the question is that it hasn't because the people that we worked with the creative residents. Only two of them were actually appointed in in universities where that would be necessary and both of them came from media arts world so this is kind of par for the course for their kind of work. So what we're doing so that was that was an affordance that we knew so what we did is started to think about if we were working with a historian, for example, what what would need to be documented to make the case for this to be in a promotion binder. And that's where the documentation as a primary scholarly output came from is we were we were utilizing the artists and the media scholars that we worked with to think about a way that we could document these projects that might be helpful for the the next historian or chemist or someone who's on the tenure track who needs to make a case for this. So we've developed a sense of a broad documentation and also contextualization thinking about the public engagement of these works. And we know now that publicly engaged works still aren't really rising on a person CV or being included in promotion binders in the same way as a peer reviewed publication. But the more we put them on those, the more that I think that they will start to. Yeah, they'll start to be recognized in different ways. Great. Thank you. Thank you, Micah and thank you for that question. We have another question now that comes from another anonymous questioner. Are you looking at best practices of documenting social practice or community engaged art practices as one way to capture this work. Do you want to take it Micah? Sure, you go first. I have a thought book ahead. Okay, so I think, yes, in some aspects, this has definitely been a learning experience and so unfortunately some of the situations we were not able to capture in the moment and I think figuring out is there a way to translate that to something that is considerable, even without the physical person there but then some of the components of the projects had built in things that were more easy to capture and so it was easier to start thinking through some of those practices for community engaged art pieces surrounding their engagement type events. I think that we do have a lot to learn and I think that that art practices are an area that's really helpful for us to have an artist join us that is in a scholarly sphere because she was able to explain a lot of those practices to us. Just like Micah was saying about giving credit for that work. Yeah, Micah, I'm sure you have more thoughts on this than I do. So, in our documentation hub, one of the documents that we worked on was these guidelines for how to talk about this kind of work, especially for promotion and tenure. So we worked with a colleague named Dr. Abby Mann who's a student at our UNC Chapel Hill Library School, whatever it's called. Anyway, what she found in kind of addressing the question directly is that there is actually are a lot of guidelines in performance art, and I'd include social practice art on a side of performance art there that will be really, really helpful for us in thinking about these things as as happenings or experiences that are meant to be ephemeral and finding ways to document them, borrowing from those established practices in the performance arts and theater, or in things like social practice. So we've written some of that into the documentation and I think we're anxious for the next resident or the next person who does a version of this project to utilize some of that documentation and see how, yeah, how it's applied in different ways. Great. That's interesting. And thank you very much for that question. And we do have a little more time if people have questions just go ahead and type them in the Q&A box there or feel free to chat it out. I have one chat that wants to thank you for the great talk and we have another question that's just come in. Could you elaborate a little bit more about conversations with the UNC Press. I would love to. They are just conversations. So the director of the press John Sharer and I have been colleagues for a while and we tend to be in the same rooms often thinking about big ideas and so he actually invited me over to the press early January to do a presentation for that for for his colleagues at the press about these kinds of works that are a really particular and unique kind of research output that that especially our library NC State is involved in that isn't necessarily reflect reflected in the same way across the rest of our library system. So it's it's just a conversation but they're intrigued, I'll say by by the challenges that this represents. And I think that we have a lot of good knowledge and models from a lot of the digital publishing that's happening and you know University of Michigan Press Stanford has their digital imprint and Brown at the same time. As we're involved with them has a separate grant working on digital publishing. So there's a lot we know about digital publishing already, but I think most of what we know about it right now is focused on the thing that we have called a monograph for a long time. And the things that we're creating are not that. So we're we're having these really. Yeah, the high level early just discussions with the press about well, what is this what does this look like UNC Press is our press also. What would it look like for us to establish some sort of a partnership with them for that lending that that stamp of certification which which is valuable in this in you know raising those things up on someone's tenure and promotion binder. Oh, that's really fascinating. Looking forward to hearing what's going to come out of those conversations. Thanks. Thanks for that question and thanks for that great answer Micah. Well it looks like we are close to the end of our time here. If you do have any more questions, please feel free to type them in now. I'm just going to share out with everyone the direct link to the program for the rest of CNI's virtual spring 2020 membership meeting so you can take a look at that link to find out what else we have coming we have a full month left of webinars. And we hope that you will join us for more and we appreciate you spending time with us here today. Seeing no more questions pop up in the in the box now I will propose to turn off the recording and end the public portion of this presentation and just invite any attendees who are still with us if you're interested in sort of approaching the podium and having a little chat with Shelby and Micah we'll hang around here a little bit longer. Just if you just raise your virtual hand I'll know that you want me to turn your microphone on I'd be happy to do that. And just check out with them some more about what they've been working on and if you've got projects you're thinking about implementing at your institution. This would be a good time to come up and chat with them so on behalf of CNI. Thank you so much for joining us. Thank you Shelby and Micah and be well everyone. Yes thank you all.