 get started. Welcome, everybody. Thanks for joining us. I'm Cliff Lynch, the director of the Coalition for Networked Information, and you've joined us for one of the project briefing sessions on the second day of week two of our virtual fall 2020 member meeting. A couple of mechanical things. The session is being recorded and the recording will be available later. There's closed captioning available if you'd like to turn that on. There is a chat box and feel free to use that. You're welcome to introduce yourself or make comments as we go along. There is also a Q&A tool at the bottom of your screen. You can use that at any point during the presentations to queue up questions. We'll take all the questions after we've heard both presentations today and Diane Goldenberg Hart from CNI will be moderating the Q&A session. I'd also note that in this, the second themed week which focuses on organizational and professional transformation, not only do we have scheduled sessions but we've also released some pre-recorded videos and I'd invite you to take advantage of those as well. With that, let me go ahead and introduce our speakers, Jeffrey Oliver and Megan Sensenze from the University of Arizona. This is a really interesting topic and actually is a wonderful compliment to the session we just had which was also about building up data science literacy and data literacy. So many of us are trying in different ways. The Clear Fellows come to mind as another example of this, of how we can really build up a much deeper pool of expertise in data science and connect that up to the broader move towards digital scholarship that's already underway in so many of our institutions. And here today we'll hear about an effort at the University of Arizona to find ways to do this. Unlike the presentation that we just heard, my understanding is that this effort focuses more on graduate students as opposed to large-scale undergraduate outreach. And with that, let me just thank Megan and Jeffrey for agreeing to come do this. They are both doing a major endurance run today because they'll both be participating in another session at four on a rather different topic. And I'm very grateful for their participation. So with that, over to Megan who will start the talks. Thank you, Cliff. So as Cliff mentioned, we already know that across disciplines, interest in digital and data intensive scholars is growing really rapidly. Many of the technological and the computational and the digital resources that we are using are living at a pace so rapid that traditional college curricula cannot keep up. So this has created a largely unmet need for critical training and technical skills on university campuses. And many emerging scholars are being left to learn on their own. Without sufficient support, these critical training gaps can dissuade graduate students from pursuing important research projects or they can create barriers to project completion. We see this in literature and we also hear it directly from our own graduate students. A parallel challenge that's also emerging is when we look at graduate student training in instructional skills and pedagogical best practices. Outside the field of education, approaches to preparing graduate students to become effective teachers are often uneven or even ad hoc. And it's entirely possible for a student to complete their degree with absolutely no dedicated instruction and training. So we know that a number of efforts to address technical training gaps have been emerging within academic libraries. Some of these include really robust engagement with software and data and library carpentries initiatives as well as programs that are building capacity through workforce development. And two really good examples of those are the leads program at Drexel and the NCSU's data and visualization institute for librarians. Yet even as we're working to build capacity, we're keenly aware we are being outpaced by change. So how can we leverage our existing expertise to meet changing in a responsive and dynamic way? At the University of Arizona libraries, we started piloting a program that are designed to address three known challenges in contemporary higher education simultaneously. These are insufficient training for data intensive computational research within a subject discipline or within several subject disciplines. That uneven attention to digital pedagogy and specifically strategies for teaching technical concepts within graduates training programs and a need for ongoing development of modular curricula that can be integrated into a classroom or used in extracurricular workshops or used independently by scholars and researchers at the point of need. We all are programmed the digital scholarship and data science fellowship and here's how it works. We focus on some core strengths among our library faculty. These include data literacy, instructional design, digital pedagogy and open educational resources. We design a year-long program that acknowledges how many graduate students are going it alone so that they can develop the technical skills they require for dissertation work. And we take a cohort-based approach to building a community of support for those students. In return, the students in our fellowship cohort design and deliver technical skills workshops for the campus community and they create an open educational resource that we can adapt and reuse to regularly refresh workshop offerings through the libraries. Alternately, they could also be integrated as modules into relevant courses within their primary disciplines. Okay, so what's in it for us? Well, we've been building internal collaboration by working across multiple library departments to establish a one semester training program led by librarians in instruction in pedagogy. We're using a competitive call for applications and that call provides really rich insights into the current technical needs of our graduates that inform other aspects of our work. We're also cultivating informal networks with multiple departments on campus through this engagement with our student fellows, which has some intrinsic benefits beyond the program. We're establishing a fresh campus-wide program of workshops designed by the fellows based on that demonstrated need. And in the process, our team is exposed to the skills and methods and techniques that we're learning from the students and that we get to incorporate into our program. And for the fellows, they get a small stipend, maybe not three dollar signs worth more like one dollar sign worth. And they also get support for pursuing the self-directed learning that they really need for their research and would be doing anyway without support. We leverage our existing campus networks to provide some additional support for their technical skill building. And they're given strategies for managing their projects, scaffolding their learning plans, and formal training in instruction. So they're given an opportunity to hone their new skills by developing and delivering an open workshop and they come away with a portfolio worthy demonstration of their efforts. And that takes the form of an open educational resource that they can link from their CVE. So in sum, they make valuable progress toward their dissertation and they leave with tangible evidence of their technical knowledge and their approach to teaching. And what did it take to get this off the ground? We started talking about it in spring and summer of 2019. Jeff, our co-author Jen and I started by sketching out a couple different variations of what a year-long program might look like. And then we started discussing that back of the envelope set of ideas with our colleagues who we had identified as prospective collaborators. With a little bit of tweaking, we came up with a pretty solid idea for a program, which Jeff will describe in more detail shortly. And next we had to come up with the money. We had originally budgeted for four to five fellows with stipends of $1,500 each. And we also included additional funds for OCAD like workshop refreshments, which we didn't end up using because everything shifted online. But one thing that was really interesting is we're at a state school and we can't use state appropriations for student stipends. So we ended up having to do a little bit of extra legwork with our administrative leadership and our business office to ensure that those funds could come from the right sources. And I see Sarah's with us today. So thanks for your help with that, Sarah. With our collaboration on bull and our funding secured, we were on to recruitment. And this phase of the pilot began in fall of 2019. We created our recruitment plan, we released our call for applications, and we offered informational sessions for potential applicants in the month of September. As part of the initial wave of activity, we also established our internal evaluation protocols and we created a shared rubric for the project team to review the applications, which were due in, I think it was October 15. So we received a total overall of 27 applications from students from all over campus. So a fully wide range that included the humanities and the social sciences, the life and the physical sciences as well. Then throughout November, our team ended up completing a review and ranking process that was followed by joint deliberation to ensure that each individual applicant met the review criteria, and also that our cohort as a whole was sufficiently diverse. We sought diversity not only across disciplines and technical skills, but also demographic diversity. We ultimately selected those, which resulted in an 18% acceptance rate. One of our fellows had to withdraw, unfortunately, in the very early weeks of the program. And so we ended up with a final cohort of four. And I will hand the next session over to Jeff, who will say more about the program and our work with the fellows. Thanks, Megan. So the first cohort, we really sort of designed the cohort thinking about selection and participation to really address those three gaps that Megan introduced earlier, the data and computational training across disciplines that many graduate students are in need of, but don't have access to. Also to address the training gap in pedagogical best practices. And then also the gap in learning materials for a landscape of ever accelerating technological advances. Like that next slide. So the applicants applied, as Megan said, they had to identify some tool or resource that they needed to use to accomplish their work as a graduate student, as well as why they needed it. And then finally, we had them describe sort of where they're coming from and their background so that we could evaluate the entire potential cohort that we were building. And the next slide, we'll see how we actually did that. So the question was for these tools or resources that the graduate students identified, was there a broader campus need? So was this something that we could see a workshop being useful to campus rather than maybe two or three other people. We also assessed whether the students were at the appropriate stage of their careers. So we weren't looking for folks who were almost done. We wanted folks to actually have a learning opportunity here. And we didn't want folks necessarily who were right at the beginning who maybe didn't know where they were going yet. So those were two criteria we used to evaluate those applications. And then finally, we were really keen to get a diverse cohort because we really wanted to see opportunities for peer-to-peer learning with this cohort. And we thought that a diversity of demographics and a diversity of disciplines representing the cohort would go a long ways towards accomplishing that. So the next slide will show the diversity that we saw. So we had, as Megan mentioned, five fellows at the beginning. The applicant pool was skewed towards the sciences, especially towards social sciences. And our ultimate cohort of five that we selected, we think was representative of the actual applicant pool that we received applicants for. And so the next slide kicks off the first semester. So this spanned over two semesters. The first one was really preparation, so preparing the fellows. We had a series of library and led instructional sessions. And we met every week for two hours. And this was combinations of instructional sessions as well as opportunities for self-directed learning using a Pomodoro technique, sort of a concentrated heads down for 25 minutes, followed by opportunities to rest, regenerate, and talk amongst the cells and ask questions of folks. So the next slide shows the two gaps that we were addressing in this first semester. So the training for the data incentive, intensive computational research, this was largely performed by the fellows themselves through some self-directed learning. And this addressed the gap in part because at this point, the only ones benefiting from this are the fellows. And we'll talk more about how we did that in a moment. And then also dealing with the training gap in how do we actually train others in digital and technological concepts. And we had a suite of our library colleagues who came in and actually helped out with that program. So the next slide shows us the start of the self-directed learning, a session where we really started coaching and providing the fellows with the tools that they were going to need in order to actually basically teach themselves, not in a vacuum, but using the resources that are available to teach themselves the tool that they wanted to ultimately provide a workshop on. And so we talked about things like goal-setting and growth mindsets, as well as some resources for managing such a learning process. I want to mention also that a lot of the material we used was borrowed from the Carpentries Instructor Training, which has a really nice suite of materials for pedagogical best practices. The next slide, we also engage quite a bit with our partners in the library. So Avon is really an expert in instructional design and especially inclusive instructional design. And so she provided a nice workshop on how we can use the concept of personas, so our audience, personifying our audience to better understand how we can deliver the material. And then other top concepts like guided learning, as opposed to sort of free-for-all learning, as well as how do we actually do useful feedback and assessment in designing materials that are accessible to broader audiences. This dovetailed nicely to the next session that we had, which was led by Leslie Salt, who talked about actual digital pedagogy. So we're in this online, this digital modality. How do we actually develop lessons for this? And Leslie provided a really nice workshop on this, on how do we develop lessons? How do we evaluate the resources that are available to us? And then really bringing in the idea of learning outcomes. And this is actually a theme that we've seen through a variety of the sessions is using these learning outcomes as sort of the starting point. So using Bloom's taxonomy to identify where we want to be at the end and then working backwards from there. And then the next session, we had our open educational resources expert Cheryl come in and talked about things like copyright and open educational resources. So if we want to share our work and make it available, as we do with this program, to make just-in-time modules or lessons that can be used in classes or elsewhere, we have to make sure that we either license them or copyright them appropriately so that they can be used by others. And so Cheryl provided a very nice workshop on those sorts of issues. And then I touched a little bit on the software licensing for folks who were doing more things more on the side of software and code development. And so the next slide just sort of sums up what we did in semester one. We were really training the fellows in training. So talking about instructional design of these short-term workshops, conveying the best practice of digital pedagogy, talking about the open educational resources, and all of the materials that we used for this are all available on the open science framework, which is a nice resource for sharing educational materials. So the next slide with pivoting to semester two, so this was the fall semester, where fellows get to now apply what they learned in the first semester. So they developed the actual workshops. We still held weekly meetings, so we cut them down to a single hour because we can give everybody an hour less on Zoom, that's a good thing, but still use the Pomodoro technique for heads down research. So the next slide we showed the two gaps we addressed in the second semester was to further address the training gap for data intensive and computational tools through these workshops. So now we have not just the fellows benefiting, but actually the entire campus benefiting from these workshops. And then we also have the fellows that are developing these modular lessons that are available online and for the most part open for others to reuse and to remix. The next slide, I will say though that we did meet over the summer. I don't know that we'd initially planned this, but it was useful for just maintaining momentum. We only met once a month, but it was good to check in with all of the fellows. And I think that it was important for, not necessarily for us to see where they were, but to provide an opportunity for feedback, weren't quite sure. And again, because the landscape changed, some of the fellows just asked for a little bit of help for adapting to the new landscape. And they did a really good job of adapting to that. So the next slide was the fellows developing their actual workshops. And again, this is where we came back to this idea of starting at the end. So starting with the learning objectives, what is it that fellows wanted participants to get out of a workshop? And then defining the audience, who exactly is going to be in the workshop and the outcomes of these workshops? And then working back towards from there. And while I put in parentheses and made the workshop, there's a lot of work that goes into that. But I think that the first three steps are critical in successfully developing a workshop that's going to have positive learning outcomes. And the next slide, I want to pause for just a moment to call out somebody on the team, Team Johnson, because the coordination support is so critical. And I don't, I certainly didn't recognize how important it was going to be. But there is a fair amount, and there's only four fellows, but there's a lot of scheduling, a lot of surveys. And like many days, we, many things we encounter a lot of correspondence. And Tina has put in the Yeoman's effort and really, I think, really made this thing bigger and better than it would have been without her support. So I just want to note that if you're going to take something on like this, having this administrative support, coordinated support is really important. And so in the next slide, we see what actually the fellows are doing. So this is, this has been a big week this week and last week, because the, this is when the workshops are happening. We've had three so far. And they cover things like web scraping in Python, and version control with Git. So some, some sort of heavy things on the computational side. There was a really great session earlier this week about because we're all moving into this online mode and online modality, what are the ethics of transitioning research that was offline into the online ecosystem. So that was very informative. And then tomorrow we've got our final one, the data visualization and inkscape. And one thing to know is that all of these workshops, I think we capped them around 20 or 25, they all filled up, they all had waiting lists demonstrating that what these fellows are presenting really did illustrate a need that was, that we're being filled in on campus. The next slide is sort of what we're doing for the rest of the semester, which is to develop these open educational resources. A couple of them are already out there. So, so one of the fellows used Jupyter Notebook in combination with MyBinder to provide resource, a lesson that's actually available online for anyone to use. Also the GitHub pages is a nice means of communicating information. And then we're still looking at OER Commons and Open Science Framework for a couple of the other projects. And with that, I'm going to turn it over back over to Megan to wrap it up. So we, as Jeff said, we'll be wrapping up our program through the end of the calendar year. And we have the three or four fellows already having delivered their workshops. One thing I wanted to reflect on a little bit was, toward the end of the workshop on web scraping last week, one of the participants spoke up and was describing how serendipitous the session was, because his lab is looking for students who are interested in doing web scraping work. And there could be paid opportunities. And so he invited any of the attendees who would like to to follow up with him. It was really a cherry on top of what was already a fantastic workshop. But later I was talking to Jeff and I said, I didn't think it was serendipitous so much as it demonstrated that our program is actually working the way we planned it. Our fellows have become experts in these emerging technical areas and they're sharing skills and expertise back to a community that is very much in need. And Jeff being Jeff looks at me and replies, chance favors the prepared mind. So it turns out that that quote is attributed to Louis Pasteur, famous inventor of vaccines. And while we're on the topic, speaking of vaccines, one thing we definitely did not plan on or prepare for was COVID. We were only two months into the program when we all transitioned online. And yet much of our planned work had really immediate applicability and perhaps even greater urgency under these current conditions we're in. So Leslie, our digital pedagogy specialist, delivered her workshop on digital pedagogy in March just as we were all scrambling to adapt courses to the online environment. Then Cheryl ended up discussing open educational resources during a time when so many resources we rely on were suddenly rendered inaccessible because of our transition home. So we recognize also that graduate school, I mean it can be really isolating even under normal circumstances. And we were really glad to have those weekly co-working sessions for checking in with students. I think that's one of the reasons we continued through the summer was to maintain some of the social, cooperative camaraderie and value that comes from that. Megan, I think you're muted. I did mute myself. Sorry. Looking ahead to the future, we're entering a new phase of assessment and we're beginning to plan our next steps. We've been spending a lot of time reflecting on aspects of the program that were successful specifically due to our local context. Considering what it would take to replicate this program elsewhere at another institution and thinking through the logistics of what a second phase of the pilot would look like for us. So for one thing, we acknowledged that our mix of collaborators meant we were able to draw upon a pretty wide range of disciplinary backgrounds as well as technical competencies that came in handy for individual members of the cohort at different points in time. So I was able to draw on my own experiences with qualitative data analysis to work through how one of our fellows wanted to reshape that workshop to address emerging issues associated with transitioning their qualitative research to fully online contexts. And Jeff ended up drawing upon his experiences with data science to help with technical troubleshooting on a near weekly basis with each of the fellows. So we're pretty confident that the, oh, one other thing, we both of us and other members of our team also have enough pre-existing knowledge to be helpful with the tools that these fellows are teaching to be reasonably good teaching assistants and troubleshooters during the fellows workshop. So we were able to monitor chat, do things like that to keep folks from having to split their attention too much when the workshops moved fully online. We do think that the general approach that we've taken would be portable to other libraries, but that the approach itself could take a range of different forms and identifying the right collaborators is really crucial. We'd be really curious to explore with you all how this model might be enacted in different institutional contexts. And we'd be really happy to discuss that with you all during the Q&A session. Okay, that's it. Thank you. Thank you. Thanks, Megan. And thank you, Jeff. Another great example of helping to bring these skills where they're needed most. And it was really great to hear about that. I see that we are just at about time. The floor is open for questions. So please go ahead and enter your questions in the Q&A box. I just wanted to ask quickly, how did you determine what workshops the fellows would be teaching? And I apologize if I missed that. Yeah, Jeff could jump in on this too. One of the things that was really interesting is we didn't expect at the beginning of the fellowship that they would have a very clear understanding of what the workshop coming out of the thing that they're needing to learn would be. And so we spent a lot of time talking about scoping and what is the piece of what you're doing generally that is generalizable and applicable more broadly. And so we worked pretty closely throughout the first semester and into the summer as we were transitioning from learning to teach to preparing to teach on how to pull out the nugget of what they were working on for their own research and scope it for the audiences that would make sense. So that was something I think Jeff and I and Jen spent a lot of time doing and really thinking through you know, what can you cram into two hours and what feels reasonable not budding off more than you can chew. Interesting. Great, thank you very much. Well, again, as I see, we're a little past time there. I don't want to hold up our attendees any longer. So I think I'm going to thank everyone, especially our speakers for their wonderful presentation and our attendees for joining us today. Thank you all so much. I'm going to go ahead and turn off the recording, turn off the public portion of this presentation. But any attendees who have time and interest to hang around, please feel free to do so if you want to ask a question or make a comment live, just raise your hand and I'll be happy to unmute you. And with that, I wish you all a wonderful rest of your day and hope to see you back at CNI soon. Thanks. Bye bye.