 I think we can go ahead and get started then. Thank you so much, everyone, for coming to our session today, Understanding Learning Analytics, Multi-Institute Undergraduate Focus Group Study of Learning Analytics Scenarios. My name is Michael Perry. I'm the Head of Assessment and Planning at Northwestern University, and I'm joined by my colleague. Hey, everybody. My name is Kyle Jones. I'm an assistant professor at Indiana University Indianapolis in the School of Informatics and Computing in the Department of Library and Information Science. I do want to thank our other research collaborators who couldn't be here today, who've helped make all this research that we're going to share with you possible. So thank you so much to our entire team. And especially, thank you to IMLS for funding. We'll talk a little bit about the funding of this in a little bit, this project, and really making this possible. So thank you so much, IMLS. So a little bit about what we'll be talking about today. I'm going to give a brief introduction to the project. And then we're going to do some of the research that we've done and led us to this point. And then we'll be talking about the third phase of this research project, which we're learning analytic scenario focus groups. We'll talk about some emerging key findings that we found from the data as we began to analyze it. Talk about some discussion points and things that we've really found interesting and want to share with you. And then leave some time for questions and discussions at the end. We're really interested in what you all think about the scenarios that we presented and our findings so far. All right, so an introduction to the project. Why are we doing this? Well, when we started this project about four years ago now, there was little research into learning analytics and specifically about student privacy and the student perspective of those privacy issues. So as we were beginning to look into this, we found no one had really asked students what you thought about these things and tried to measure that. So with that, we came up with some guiding research questions that have carried us through this whole project. How do learning analytics initiatives align with or run counter to student expectations of privacy? And how much libraries maximize the benefits of learning analytics while still also respecting those student expectations, right? So how can we do learning analytics in a way that does respect students? Again, thank you to IMLS. We offered a proposal getting to know their data doubles. This was funded in May of 2018, just over $1,500,000 in funding. We did, because of COVID-related issues, extend the project into April of 2022. The scenarios, the focus groups, we'll talk about the move virtual, but that was slightly disrupted because of COVID. So what are the goals of this project? Really, any learning analytics practice or technology that should be informed by the student perspective, somewhere we need to be able to say, we understood what students would think about this and we accounted for that, right? So our individual goals then were to identify the reactions, both positive and negative to learning analytics, right? To understand those privacy expectations and preferences, especially amongst specific demographic groups where the data allowed. So we're really interested in what a lot of different student types think about these things. And we really wanna recommend information policy and especially technology design changes to institutional administrators, right? We wanna make sure you are all equipped with understanding what those student expectations are, right? And then also able to take that into your planning for these things. All right, so we had three research phases. I don't know why it's two. Oops, apologies. So the first phase, we conducted semi-structured interviews with students across eight institutions. Each institution did about 15 interviews. Really to understand generally what students thought about learning analytics, what their expectations of privacy were, and specifically how that related to academic libraries. Using that information, we launched phase two where we developed a survey to then field sort of at mass to undergraduate students within each of those researcher's institutions. And then finally, that brings us to phase three where we ran a series of scenario-based focus groups with students to explore the applications of learning analytics and what they thought about very real learning analytics scenarios. So phase one, as we mentioned, semi-structured interviews with undergraduate students. We really wanted to know about generally how they felt about data mining and analytics practices and how they felt about the existence of those things within higher education, and then specifically in relation to the library. Pardon me. We did write up the findings and you can find the article in the citation below with the results of those 120-ish interviews that we did with students. So we used all that information that we learned in phase one to launch phase two where we wanted to conduct a survey at each of the institutions with undergraduate students. Really to see those themes that we found in those 15 interviews, did those carry true if we looked at students in much larger numbers? The results from this one are will be published soon in library quarterly so you can look forward to that one as well. Just real quick, Mike, remind everybody what the N was for phase two. How many students did we have participate? Was it about 2,200? 20, yeah, just over 2,000. Yeah, so we had about 2,000 student responses into the survey. And that brings us to phase three, the one we really wanna talk about in depth here and share findings from with you, the learning analytics scenario focus groups. So we wanted to take those insights that we gained in those first two phases and use those to develop three future focus scenarios to have students discuss in a focus group, to really present them an actual learning analytics scenario, have them talk through it. So the goal was to conduct three focus groups at each of the seven institutions. So we did a total of 21 focus groups. And the really important thing and why we wanted to do focus groups to talk about this, we wanted the students to reach consensus around themes of trust and privacy and we'll talk about explicitly what we mean by those in a minute, how they relate to the scenario, right? So what we wanted were those focus groups to be able to come together and say this group of students came to agreement that these are the conditions that they felt this technology would be acceptable for them, right? And hopefully then we can use that to really clearly map how we develop learning analytics initiatives. So I've mentioned learning analytics a lot. What do we mean by that? You can find the traditional definition of learning analytics right there which we, everyone sort of refers to it. But the one thing we really wanted to make clear especially as we developed these scenarios. Learning analytics often is talked about in understanding learning, right? And learning outcomes and how to increase those efficiencies. But when we talked to students especially in the earlier phases one of the things that became really clear was that students think about a lot of other things that that data is going to be used for. What came up a lot was, especially as we talked about the library, optimizing research allocations and then hopefully improving institutional efficiencies, right? Can the library use data to better buy stuff so that I have access to resources? Wonderful. No, we can tie that to a learning outcome obviously as well, but it's a little bit of a different flavor than as we normally talk about it. The other one that came up a lot was safety, well-being and security. This has especially changed in response to the pandemic where a lot more data collection has happened, right? And we're now much more interested in location tracking where people are, are the right people to be there, those kind of things. But this came up a lot in both the interviews and the focus groups as well, that sense of using data to provide a secure environment for students. So a little bit more expansive than just actually measuring learning. Yeah, and just one comment on that. I mean, we see this in the library literature a lot too about how academic librarians are scoping out learning analytics is, yes, being about learning and learning outcomes and we wanna know the relationship between what students are able to gain with regard to their learning and how they're using library resources and spaces and interacting with personnel. But the library literature also suggests that that's politically valuable for the library too by knowing these things, by doing this type of analysis. And that political value also has financial implications. If we can prove our value to administrators to other stakeholders in the institution, then that will help sustain financially the library and that is a connection to learning analytics. All right, and I mentioned privacy and trust. Those are the big sort of overarching themes we wanted to talk about in these focus groups. So what do we mean by that? Well, privacy, we're talking about a student's right to have identifiable student data and information collected, stored, and utilized by the institution but for authorized purposes, right? Trust, we're talking about a student's willingness to be a vulnerable party in the relationship with this institution and those actors, right? So they have trust in the institution, trust that the data will be used, kept secure, used for their benefit. So what are the research questions that guided us as we developed these scenarios and launched this research? First and foremost, what expectations do students hold regarding trust and privacy as we just defined them in relation to the scenarios that we developed? What alterations to those scenarios affect the acceptability of the scenario, right? If you start to change parts of the scenario, does it become more acceptable to students, less acceptable? And finally, what trust and privacy conditions do students agree have to be in place for the scenario to be acceptable? Again, reaching consensus so that we can really drive those groups of students to come up with what they view as an acceptable idea. So how did we develop these scenarios? What we decided we wanted to do was futureize these scenarios. And by that we mean develop scenarios that push the boundaries of current learning analytics initiatives, but that have clear roots in stuff that is happening now, right? So not just talking about the initiatives as they're happening at our individual institutions, but really taking that and pushing it forward and looking at what learning analytics might look like in a few years after the technologies has been more developed, right? The other crucial component though was we wanted to ensure that these technologies and their impact could still be understood by the participants in the scenario or in the focus groups, right? This whole exercise wouldn't have been valuable if the scenarios became too technology focused, too speculative and students couldn't really see themselves represented in it. And then finally we wanted to create scenarios that would resonate with administrators and their visions of learning analytics at their institutions, right? And the future they saw there for those. So we really want hopefully these things to resonate with you all, right? Where you can see things you might be thinking of and how they tie into the scenarios that we created. Yeah, and just real quick on this, there's a rich methodological history of doing scenario-based focus groups in the science and technology studies. And that's the literature that we pulled from in order to develop our scenarios. And one of the underlying purposes is to position the students' perspectives and expectations of these socio-technical practices at the front or the upstream of their design so that when these socio-technical systems and practices do develop, do become mature, are actually used in libraries they're informed by the student perspective. So how did we develop these scenarios then? We had everyone in the research team basically create general ideas about what a scenario might be for this research and collected all of those VAO worksheets, we had some unified fields across all of those. We then had everyone review those scenario ideas and use some card sorting activities to understand the relation of that scenario to libraries. We wanted these to be library focused and really resonate with you all. And then also participant comprehensibility, right? We wanted to make sure again, they were really clear to students what this technology was and what it was trying to do. We then identify sort of the scenario ideas based on the potential benefit of the technology, right? What was this technology going to get for libraries? And what were the potential privacy risks there, right? So that we could then basically find three scenarios that we chose to develop for the research. Once we had those three ideas that sort of met all those criterias that we were looking at, we broke up into subteams to conduct our literature review around the topics in that scenario, outlined the justification of the scenario, goals, benefits, and potential privacy harms as we saw them. So we could really try to develop a very robust idea of the scenario before we even began to figure out how we presented to students. Sorry, apologies about that, technology. So the subteams then conducted the literature review, right, outlined all of the ideas for that scenario so that we really had a robust idea about what it was. So how did we design the scenario to be presented to the students? First, we wrote an outline of the scenario, and I'll show you what those look like for the three scenarios in a bit, but just a sort of snapshot about what this was trying to do. We then outlined exactly what it would be doing, examples of data that would be used in this process so that students had a really clear idea of what was going into these systems, what the goals of the technology were, what the institution was trying to accomplish, and what their rationale for use was, right, like why were they going about doing that? We then, this really begat the focus group portion of each of the scenarios. We started talking about trust, presented, asked them, how do you feel about trust as it relates to the scenario? Provided some alterations to the scenario where we changed whether data was identifiable or de-identified if it was used strictly for education purposes or more expansive, you know, any purposes, or if the data was being shared with just the library versus everyone at the university to try to get a sense of how did those factors change how people viewed trust? We then did basically the same thing for privacy, right? Talked about, generally, how do people feel is their privacy being respected in these scenarios? What do those alterations change about how they view privacy as it relates to that? And then finally, what is their consensus around privacy and how this technology would respect privacy? So the three scenarios that we focused on were first what we call the library management system, or really it is pushing all of the library services through the LMS so that data about the use of library services and resources can be tied together with Canvas and Blackboard data collected and then made available to analytic purposes. So really taking library use data, pairing that with data out of the LMS and making that available for librarians or other people as we altered the scenario to be able to analyze and then use hopefully to improve teaching and learning. The next scenario dealt with the development of a library data warehouse. So really this was centralizing student data across all the different systems that a library uses into a single data warehouse, which would enable librarians to know resources, which services students were using and would allow them to combine that library data with other data perhaps sourced from the larger institution like academic records or student profile information. And then really using that for analytic purposes to understand that. And the final scenario we talked about, which we won't talk about when we get to the results just due to time, was geolocation tracking, which basically the university starting to use a system that compiles and organizes geolocation data, sorry, stuff's flashing and it's throwing me off. And really taking that geolocation data and then combining that with other student data, again for analytic purposes as well. So as I mentioned originally when we were scoping out this three-year research project, the idea was to conduct all of these focus groups in person. As we all know, that was not possible because of the response to the COVID pandemic. So we needed to reconvene, reconceive this as virtual focus groups that were going to be conducted over Zoom. This meant really detailed tracking of settings and logistics for those things in dealing with seven different institutions, various Zoom setups, which can actually vary immensely when you look at how institutions have configured them. The nice thing though is that it allowed for broader participation by the researchers for the focus groups. So instead of just conducting my three and those being the only three focus groups that I witnessed, I was able to then sit in and take notes on researchers focus groups or at other institutions. So I found as a researcher that was really helped expand my understanding of how students felt about these things and helped me clear up some of the biases that might have come out just of my admittedly private and sort of outlier institution compared to some of our other research sites. And this did allow us to capture video and audio as well so that when we're analyzing these, we can go back and actually look at the video from the focus groups. We were lucky to be able to write up sort of how we did this virtual focus groups in this forthcoming piece as well which might be interested if you're looking to work on focus groups virtually. So actually getting down to conducting the focus groups. How did we do it? We got 3,000 emails from each of the institution sites and then built a Qualtrics form to allow people to sign up for focus groups. We basically sent reminders to that pool until 10 students were registered for each of the focus group times for each of the three. We set the threshold for conducting the focus groups five knowing based on some previous research and our own experience, we'd probably have drop offs and no shows for people who had signed up for that. So we would hope to do five and if we didn't have five people reschedule the focus group. We did end up having to reduce it to four due to students, one instance where a student dropped out after the focus group had already begun and no one had sort of realized that until afterward. So we completed the focus groups between March and May of 2021 with each site conducting three, 21 total. In total, we had 116 participants for the focus groups. They averaged about 40, we've scheduled them for an hour but they averaged about 44 minutes in length which is not surprising there when we look at, again, the research around doing those kind of focus groups. So all of the research documentation for the focus groups, the protocols, informed consent recruitment, everything is available in our OSF site which there's a link to at the bottom. It also includes information about how we developed those scenarios, the slide decks for those scenarios as well so you can see actually all of the information that we presented to students if you're curious and you can find a link to that right at the bottom. With that I'm gonna hand it to Kyle to talk about the emerging findings that we've discovered so far. Yeah, first off, what we wanna say is there is one caveat. This is research that is ongoing. We have done probably our first 65 to 75% of analysis but as we dig into it a little bit more we'll get some more descriptive depth or breadth. We'll get some more theoretical depth as well. Our aim here is to finish everything by the end of February and submit and then as soon as we can put a pre-print into our OSF site which is where we've deposited all of our research materials over the last three and a half, four years. You'll then be able to access our full complete findings then. So you can find out all of our information on our website at datadoubles.org. So we're gonna be talking about two of the three scenarios, the LMS and the LDW scenario and we're going to leave out the TRK, the real-time location tracking scenario. Primarily because the LMS and the LDW scenarios are directly related to academic libraries and their data capture and use practices as we have futurized them. So first off, the LMS library services embedded in the learning management system. So again, recall that we had three general areas for each of these scenarios, the general question, the alterations in the consensus and what we're seeing in the general question, the kind of reaction, if you will, to this particular scenario. We're seeing a lot more approval than disapproval as we have qualitatively coded it using MaxQDA. So we're seeing on the whole about 105 segments to 75 segments of disapproval. But of course that quantification doesn't really mean a heck of a lot until you dig into it and figure out what the qualities are of approval. So students tended to think of this scenario as being very much about, or the data being very much about educational data, about having educational purposes as being directly related to their educational experiences and the information not being very personal in nature which is very important as we start to talk about the LDW scenario next. So it was for the most part a very acceptable scenario when the data and information were tightly connected to improving the students' experiences and that the data were linked to education kind of broadly speaking. So that's important to note that students are already carving out kind of use cases that are okay with them and use cases that are not okay with them. So there was a clear understanding by students of how looking at LMS data in library data that's embedded or connected to LMS data could definitely aid students. It could improve library practices, lead to new technological advancements that could help students negotiate research practices within the library and connect those things to their learning in classrooms. So a couple of quotes here from students that kind of explain those findings. So the student says, I think it makes it more understandable what information students are using and where they are better spending their time when it comes to resources that the library provides. So when it comes to them, this being the library, using it to develop a better understanding of their student population and the information that they use, I think that's completely valid. I'm fine with that in my opinion. Similarly, a student says, it says that their main goals it being the slide deck that we presented to them in terms of the justifications for the practice. It says that their main goals are to help students to help librarians as well in the long run. So my kind of view on that is like, why not? So with regard to privacy and trust, there are some conditions here that are really important. They say that trust and privacy is respected in this scenario to the extent that data and analytics are focused on education, again educational purposes that we addressed before, that the practices are transparent and that data is unidentifiable. And this unidentifiable part is very key and it will follow again in the library data warehouse scenario. So they have trust in librarians who are accessing and using the data and more trust in them than other institutional actors, especially faculty were called out. They don't really have the same degree of trust in faculty that they're going to use these types of data and these sources of data because faculty make judgments about them, right? And librarians are not seen as staff or faculty, depending on your campus, who actually judge students and have a real say in their academic progress in terms of their grades. And this is important, I think too. A lot of the students are saying, well, up to this point, there's really no reason not to trust librarians and libraries, right? With regard to their privacy, they haven't had many privacy invasions, your breaches or conflicts, however you might want to characterize it. And so they kind of look at this and they go, well, okay, yeah, we trust libraries to do this work. But students do express some concern about the data leaking outside of this particular system and this leakage reaches attacks, however you want to characterize it, is very important to home in it on as well. Because what they're saying is that, look, there's a specific purpose for how this data is going to be used by particular actors and we're okay with that purpose and those actors, but as soon as things change, that's when we get a little concerned here. And there is some skepticism that the data uses can change to advance institutional, but not necessarily student interests, especially when libraries and institutions start to partner with third parties, which is ironic considering that our learning management system providers are third parties, right? So students aren't really seeing that connection very clearly, at least in the data that we have. So students say that, you know, I think that either based on because I've never paid attention or because it's never become a problem, the library and the university respect my privacy enough that I've never been concerned about it. I know that they have data and it's never been a problem, but I don't know how much I think companies as a whole respect my privacy. Companies are not known for caring about individuals and this completely reflects what we saw in the phase one interviews with students. They also say, you know, most people seem to believe that library and university will generally have their best interests in mind, I'm of the same opinion, but then they start to talk about third parties. So is there a consensus on what conditions are necessary for the scenario to be sustainable in the long run? So these, when we start to focus students on saying, hey, here's the goal, we've gotta develop a consensus. What are the things that you all agree on, right? This is what they say. They want granular consent. They want de-identified data by default with options to opt into identifiable data gathering. They wanna know the specific collection practices so that they can review them and have a say about them when they do consent or choose not to. They wanna know about the specific uses, the access rules considering concerning what institutional actors will be able to gain access to the data. Again, this is because of their concern with perhaps faculty gaining access to it. And they want consent reminders and multiple opportunities to change their consent choice. Again, it has to be for educational purposes. And students realize though that they need to take some personal responsibility for understanding their consent choices. They need to read any documentation that's provided by the library or the institution regarding consent. They also want to audit the data that's available to the library. And they wanna know how that data has been used by the library and by whom and for what purposes. They wanna kind of keep a trail or be able to track down that trail of data usage. And they want personal access to their own data. They obviously, time and again, say their own data in kind of a property sense, right? That's something that we can discuss. Do students actually own data that is collected or created by the institution? That's actually a very controversial point. They say if it was unidentifiable and couldn't be tracked to be, then maybe that would be okay. Unfortunately due to time, I'm gonna skip through some of these a little bit because I do wanna make time for discussion here. So the library data warehouse scenario. So one of the things that I wanna point out here is that when thinking about the LDW scenario, think of it as an iteration on the LMS, right? The LDW, the library data warehouse scenario introduces more data sources that data is more granular and data are now aggregated across multiple systems and are more broadly shared and made analyzable to other institutional actors. So we're kind of expanding the scope here if you will. And as we put into the scenario, you're identifiable by default. Like that is the purpose of this data warehouse to identify you to analyze data about you. So do students approve of the scenario? Well, according to the coded segments, again, quantification is a limited measure here, but there's a little bit more disapproval, or sorry, more disapproval than approval, but it's a pretty even split when you just look at the segments. And like with the LMS scenario, students saw how that scope was enlargent with the scenario, with the LDW scenario in comparison with the LMS one. They saw how their behaviors could be captured in a much more detailed way. But they also recognized how it could be beneficial for education kind of writ large to do this type of analysis. And they saw some logic behind collecting the data to improve library services and resources, but they had some concerns. So this student says, it sort of rubs me the wrong way a little bit. I think that there could definitely be some helpful services offered based on what I've checked out or what I've asked about that's captured in the warehouse here. But I don't know. I think any like revealing information about time or location, I don't know, makes me uncomfortable. Feels like a weird kind of tracking thing. Student says, I feel like there are other tools that they could use to get the information that they want that doesn't involve just collecting large amounts of data from students. And the other student says, but if it was just turning over all data collected without like a specific reason, then I don't think I would trust that. So do students think this scenario respects their privacy and trust? Again, students express clear trust in libraries here. And there was an obvious sentiment that librarians care about students. Libraries have students interests in mind in that they want to improve the learning outcomes for students, which they can see a connection between the data warehouse work in creating programs and services from the analysis of that data that can improve their education. But, and I thought this was very interesting, the students do realize that this is just data fishing in a lot of ways. It's just collecting up all the data that's available for possible uses, some of which are unclear to students. It could be unnecessarily broad and that needs a justification. And there are concerns again about wider access. So with the LMS, they were concerned about faculty. Now there's much wider access in terms of institutional actors and they kind of homed in on that in terms of their privacy concerns. Couple of quotes here. I do trust them, them being the library because I mean libraries are supposed to help you. It's supposed to be something safe for the students. They don't, the student says, I don't see anything wrong with collecting my information and I trust the library generally. And if it's a university library, I trust them even more. Within the library then, I don't think I would have an issue with the information being shared. Again, the scoping here is important. Within the library, that's the key term. And this other student says similar, it's just within the library, so I would trust it. They start to segment the data sources out and they say okay, or the data types and they say okay, but it's getting very personal information here and I am not necessarily comfortable with that. And the student says, I feel that if our personal information, such as location, phone number and name and even email address is collected by the library, university or companies, I think that means that our privacy is clearly not respected. We could have a conversation with the student about all the various definitions of privacy, but this is the student's perspective. So what are the conditions that make this sustainable? Again, granular consent came up, auditing mechanisms, access to one's data, just like with the LMS scenario. But for the LDW scenario, the bottom line was if consent is not an option, then de-identification needs to be the standard practice. Well, a lot of us or probably all of us in the room know that de-identification, especially when you have a lot of different data sources and types, is very hard to protect against re-identification. They wanna be consulted, they wanna have agency, they wanna have some choice making in what's going on with this library data warehouse and that they want the consent mechanism, again, made available to them, but especially to make that consent mechanism intelligible in short, not a Facebook terms of service thing, that's not what we're talking about here. They want something that is in a couple of pages or so that they can read and really take action on. And they wanted notifications about when the data in the warehouse was used and for what ends. And they wanted more justifications from the library as to why they're collecting the data. A couple of students also stated a desire to expunge data post-graduation, which I think is very interesting. They wanna delete their data trail as it were. Of course, that's problematic when you need historical data to do some types of analyses. So again, some of these quotes just reiterate what I just said. They want the communication from the library. They want that to be very clear and not just to do it once and expect people to really remember, remind them of what you're doing with the data. Remind them that they have a choice if you give them a choice to consent. Again, show them exactly what's being tracked and how much. And now I wanna get into some discussion points and hopefully we have a few minutes for Q and A for you all too. So one of the things that we're talking about in the research team is that libraries have the savings of trust. Perhaps a larger savings of trust than the institution does and especially more than companies and other third parties do. And what happens here is that with the savings of trust, if you break that trust, of course you lose that savings. You don't have that goodwill anymore and that's problematic. Mike and I were talking about, we feel like if you lose that savings of trust, it's gonna have knock-on effects or other services that you offer to students. To what extent will students be willing then to come into the library or engage with reference librarians or go to information literacy sessions if you've had a massive data breach from your library data warehouse. It's just one scenario. Again, this educational purpose point coming from the students is really important, especially given what we've seen in the literature about how to use this data for political or financial gains from the library or even the institution. And students really key in on that and they're just not all right with that, right? So the justifications are not, we're using your data to make money, we're using your data to improve necessarily our cost savings and a purchase of some type of database or material. They want to see how you're connecting your data collection and use and analysis to actual educational outcomes. And since they want to be consented, obviously they want to engage in your practices. They want to have a say in what is okay and what is not okay and to have in some ways a discussion with you if you're pursuing learning analytics and especially these two types of scenarios. And that's not autonomy concern. Finally, it comes down to consent. That's what this comes down to. We did not say consent in any of our scenarios. We did not prime them to say consent. They said consent. They engaged in that discussion. Alls we did was say, well, what do you mean by consent? What are the conditions around consent that you want? The problem is that historically, higher education institutions and the libraries within those institutions do not have a way to enable consent. There's very little technological infrastructure built up to enable consent. And the way we practice consent is typically related to research. And in the case of the evaluation and assessment practices that we do within a library, that's not traditional research that requires consent in a lot of ways. We do a lot of all of this information gathering within the institution already without asking students if they're okay with it and what kind of rights they feel like they should have to it. We have no, so we don't have the historical background. We don't have the technological infrastructure. We don't really have the legal motivation to do this. I don't know if we have the political motivation to do consent because that could reduce the access to data that we may want within the institution and that's problematic. So my question, and I would love to hear some feedback on this because I'm kind of stuck at this point, is like what motivates you all to actually pursue a consent mechanism for students? How does that look? Is that practical? That's really important because after three and a half years I think this is what it comes down to in this research is that students want to be consented and want to talk about this. Now the question's on us as librarians and administrators and institutional leaders, how are we going to respond to this? So just some guided questions for you, feel free to answer these or ask us anything you may have but we have some time to talk and of course Mike and I are definitely willing to talk after this session if you'd like, thank you. So there's a, thank you. And yes, there's a mic in the middle if you could use that if you have questions. Hey, thank you so much for the research. So two things, one is an observation which is with most of our institutional privacy policies I think we are already in a blanket way that most students probably never look at saying we will collect your data in order to improve your experience in our programs. So there's a sort of like, how do we educate students about the fact that that's happening? And then my other thought is really around what you flag in the focus groups around mistrust of the faculty teaching the courses and that seems like a deeper cultural challenge that needs to be resolved and I just would welcome any reflections you have about that or that power dynamic is where students are feeling that if that relationship isn't trust based then probably that's the lynch pin and anything else that's happening around that relationship is more tenuous. So I'd welcome just what your reflections have been. I think one of the big things with the fact and the notion that they sort of trusted librarians trusted us more than faculty. And I think it came up especially as we talked with outside of the institution as well sharing it students really do not want their data to be used as a mechanism to judge and or punish them. And in particular one that keeps coming back to me at my institution we had one where a student was re-identified by a photo for participating in the student protest and that became a non-starter for them, nothing the university could ever do would reinstate that trust that they had that was violated there. So I think and as Kyle kind of said the fact that librarians are viewed as these sort of neutral parties for lack of a better term I don't know that I fully agree with that. We're not gonna judge them. So they immediately defaulted to a well you probably want that to improve something, right? Buy better books, more appropriate books, better databases, those things. And the door wasn't really open for it to become a sort of mechanism to evaluate. Yeah, I mean coming out, we're not out of the pandemic let's be clear about this but getting on the other side of some things I wonder if students really should trust us anymore. I mean Cliff touched on this with the online proctoring. We have as institutions put students in harmful positions, not all of institutions but put them in harmful positions by forcing them to engage with particular technological infrastructure and artifacts. So I think we're already losing some of that trust coming on this other side of the pandemic in libraries, you're in a good place. Faculty like myself, I'm an LIS online educator. I feel like I have to do more to earn that trust back and say this is how I'm gonna treat you, this is what I've done to protect you. And I think perhaps more of our faculty need to do that. Thanks for the presentation and the research and the framing around consent which I think is really, really interesting. I'm struggling a little bit with how to, you've sort of framed out a kind of moral imperative here. And I'm struggling a little bit with how to put that into dialogue with the way that institutional alignment works in like actual organizational contexts. So there's a lot of discussion about how libraries need to be in alignment with the strategic directions that their universities are moving in. And as you know, in some cases anyway, universities are moving in the direction of, for example, carrying deeply about student success, retention, graduation, progression, et cetera. And for better or for worse, one of the tools that they've, some institutions have chosen to use is a sort of learning analytics framework, right? So I mean, and students may or may not want to be monitored through those platforms as I think you've done a nice job of articulating. So how does the library in that scenario engage, right? The provost is saying, the president is saying, whoever, you know, the number one priority or one of our top three priorities is retention, progression, graduation. We've adopted this strategic direction, learning analytics, et cetera. And I've heard library directors, not many, but a few, in that kind of an organizational context say, I'm not gonna do it, okay? And I can think of at least one case where that person is no longer employed as a library director, okay? So I guess what I'm wondering is like, can you put this kind of moral imperative into a bit of a sort of organizational framework of that sort and help us understand what you're calling for in that kind of circumstance? Thank you. Yeah, I think one of the first things I want to acknowledge, there are definitely power dynamics that are at play and often libraries are going to be profoundly disadvantaged if the university wants to pursue learning analytics as a way of justifying to, again, places they might have to be that like a state legislature for schools, right? The library doesn't have a lot of power to bring in those sort of negotiations. And as you said, you could say no and you could just as easily find yourself outside of a job. So I think it's really important that we frame, we take our professional values and the things that we think are really important for librarians. And we're able to articulate those and show those, show how those values tie to like research like this, right? And how we would react to these things and understand that we may still be forced to participate in that, hand over data and do those kind of things. But I think at a minimum, we can always guarantee we position ourselves as experts when it comes to things like student privacy, right? Really engaging students and really becoming sort of partners with them with that. So even if we're as institutions sort of strong armed as it were into participating in large scale data, aggregation and analysis, I think we can still at least bring these issues to bear for the institution. And that might not change the needle, but I think it at least affords us an opportunity to have a dialogue around those things. Yeah, it's a difficult question. Bringing in the morality of learning analytics and trying to tie that into the mission of the institution is difficult when the moral center of many higher education institutions has disintegrated in a lot of ways to be just focused on retention and graduation and success metrics that are more financially and politically aligned than they are having to do with educational learning and outcomes, right? Like retention is a proxy. It doesn't actually tell you if anybody has learned anything. So the question I would push or the way I would push back on that is I would try to get the administrative team or whoever's bringing that to the table to say, okay, well, what do you mean by retention? Yes, it's a quantification of something, but is it retention for what goal, what outcome, right? And if it is about student learning, then we don't want to change the conditions around learning using data practices that change students' behaviors, puts them into like a chilled type of experience where they don't feel like they can actually openly engage with their peers and their faculty because they're being data, data-fied, right? We want them, if that is the concern, then consent is one way to kind of break that down because we're actually creating a discussion and students are willing to be data-fied in some ways because they've said that's okay. And then you don't have some of the harms that maybe you would otherwise. I don't know if that makes a heck of a lot of sense, but I just feel like consent is the way to create morality around the data practices even if the data practices are for financial purposes. Yeah, I think a lot about how my institution, which is a private institution, retention and GPA are not things that my institution looks at or cares about. I understand completely why they do for other places. That's taking the form more with the focus on the student experience and student well-being as another one, and those are things that, especially as we are still in the midst of the pandemic, institutions are caring deeply about and become really important. And I think that might be another way if we can shift the conversation from just those metrics like retention and GPA to something deeper and bigger, that the experience and the well-being of those students, that also opens the door to talk about these things in a different way. Because you can talk about, sure, we could possibly learn things out of these analytic practices, right? But we also could do potential harm to students by data-fying them, by having that chilling effect, or just potentially by collecting data that could be exposed in a data breach to very real-world ramifications. So I think that might be another way to kind of change the frame of the conversation to get at these deeper issues. Yeah, I just think consent is care. I think that's what it comes down to. If you give students an opportunity to consent to these practices, it demonstrates that you actually care about them as individuals, as human beings who have rights, right? And I think that's going to drive up that sense of trust that people have both in libraries and in those institutions. And part of what we do is not only to provide an education to people, but is to provide, you know, to be part of something, right? To be part of that institution, to be part of that institution's history and what it does going forward, those are crucially important. So I think that's really, really important. I'll just say, one of the threads that you just kind of both spoke to a little bit is the fact that some institutions have different dynamics than others, right? So in the case where it's all about the student experience, you know, maybe the benefit of data-fying someone or the threshold for consent for that person might be different than in a school where the implicit, the argued trade is, hey, we're gonna increase the graduation rate from 32% to 65%, is it worth it to you in that case? Right, you might have a different kind of focus outcome at that school than at a different kind of school. So anyone sit down, but thank you so much for that. Oh yeah, no, and I think that's a real good point too. And something that I think we saw in some of the more, I would say like, where you had focus groups or participants had clearly thought about these issues. Some, a lot of times the sort of lack of government oversight of this and an acknowledgement that absent that, maybe one institution can change, but really without that wholesale, you're gonna run into those problems where each institution has slightly different cultures and slightly different expectations and looking to achieve different things. All right, we'll end it there. Thank you so much. Thank you so much, everyone.